Neuroscience Research Notes
ISSN: 2576-828X
OPEN ACCESS | RESEARCH NOTES
Test-retest reliability of the questionnaire on the screening of sleep disorders
Enkhzaya Batkhuyag 1, Enkhnaran Tumurbaatar 2,3, Battuvshin Lkhagvasuren 2,4, Enkh-Uchral Perenleisambuu 3, Enkhjin Bat-Erdene 3, Myagmartseren Dashtseren 5, Otgontuya Duurenjargal 6, Xihua Zeng 1*, Gantsetseg Tumur-Ochir 6#
1 Department of Psychology, School of Public Health, Southern Medical University, Guangzhou 510515, China
2 Department of International Cyber Education, Graduate School, Mongolian National University of Medical Sciences, Ulaanbaatar 14210, Mongolia
3 Brain Science Institute, Ulaanbaatar 14210, Mongolia
4 Department of Psychosomatic Medicine, International University of Health and Welfare Narita Hospital, 852 Hatakeda, Narita, Chiba 286-8520, Japan
5 Department of Family medicine, Mongolian National University of Medical Sciences, Ulaanbaatar 14210, Mongolia
6 Department of Mental Health, School of Medicine, Mongolian National University of Medical Sciences, Ulaanbaatar 14210, Mongolia
# Correspondence: e-mail: gantsetseg.t@mnums.edu.mn; Tel.:+976-9906-4741
* Correspondence: e-mail: 58750789@qq.com; Tel.: +86-20-62789530
Received: 1 July 2021; Accepted: 15 September 2021; Published: 26 September 2021
Edited by: Jambaldorj Jamiyansuren (Mongolian National University of Medical Sciences, Mongolia)
Reviewed by: Yusuf Yusha’u (Ahmadu Bello University, Nigeria); Tan Hui-Jan (Universiti Kebangsaan Malaysia, Malaysia); MS Zobaer (The University of Texas Health Science Center at Houston, Texas, USA)
https://doi.org/10.31117/neuroscirn.v4i3Suppl.90
ABSTRACT
Sleep is a vital component for maintaining good physical and mental health. When sleep is disordered, it can adversely affect an individual’s mental and physical wellbeing. Although a variety of single specific disorder measurements are available for specialized clinicians use across the globe, to date, there are no proper screening instruments available for screening multiple sleep disorders at the primary care level. The purpose of this study is to determine the test-retest reliability of the sleep disorder screening questionnaire (SDSQ) for detecting multiple sleep disorders among the Mongolian population. A total of 366 university students were recruited for participation. SDSQ was administrated to all participants on two occasions two weeks apart. The test-retest reliability intra-class correlation (ICC) and Cronbach’s alpha of the SDSQ were calculated. The sample consisted of 194 (53%) females, and 172 (47%) males, with a mean age of 21.6. Observed test-retest reliability was ranged from 0.53 to 0.81. The SDSQ was found to have excellent internal consistency with an alpha=0.950. The external validity of SDSQ revealed strong test-retest reliability in the current study population. The discriminant and convergent validity studies are required for the further improvement of the construct of the SDSQ.
Keywords: insomnia; multiple sleep disorder; student; scale;
©2021 by Batkhuya et al. for use and distribution according to the Creative Commons Attribution (CC BY-NC 4.0) license (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited.
1.0 INTRODUCTION
Sleep is a vital component for maintaining good physical and mental health and fundamental to healthy living. Inadequate sleep leads subjects to be affected by a series of diseases and disorders that contributes to increased social and economic (socioeconomic) costs over the years (Sleep Health Foundation, 2021). When sleep is disordered, various health problems such as cognitive performance, mental resilience, metabolic and cardiovascular health, and workplace productivity and safety will be affected. Moreover, it can adversely affect the quality of life. For instance, obstructive sleep apnoea (OSA) has been associated with hypertension, diabetes, and depression, while insomnia was associated with mental disorders features such as generalized anxiety disorder and major depressive disorder (Ohayon, 1997). Sleep disturbance leads to obesity and obesity-related complication (Muscogiuri et al., 2019), as well depression and anxiety were strongly associated with insomnia in the Qatar population (Khaled et al., 2021). The study among the Korean community found that sleep disorders among patients with type 2 diabetes increase the risk of dementia compared to those without sleep disorders (Jin & Choi, 2021).
Over the years, followed by technological development and rapid urbanization, the prevalence of sleep disorders has increased steadily. For instance, in the world, there is one-third of adults suffer from some form of insomnia (Sleep Health Foundation, 2021). In the United States, every year, 50-70 million adults are affected by sleep disorders (National Institutes of Health, 2011), whereas in the European Union (EU), approximately 1 in 2,000 people are affected by narcolepsy (Sleep Health Foundation, 2021). The study carried out in the French community reported that 18.6% of the general population had insomnia complaints (Ohayon, 1997). The study conducted in Madrid, Spain (Vela-Bueno et al., 1999) revealed a prevalence rate of 11.3% of insomnia, 11.6% of daytime sleepiness, and 3.2% with hypersomnia. The prevalence of general sleep disturbances (GSD) in the Netherland study was 32.1% (Kerkhof, 2017). The reported prevalence of sleep disorders in Turkey was insomnia 15.3%, excessive daytime sleepiness (EDS) 5.4%, restless leg syndrome (RLS) 5.2% (Demir et al., 2015). Upon the Chinese population, the prevalence rate of insomnia in the general population was 15.0%, lower than those reported in western countries (Cao et al., 2017). Nonetheless, the prevalence rate among university students was high as 23.6% (Li et al., 2018).
For these reasons, sleep assessment is an essential component of any health check. A variety of measurements for a single specific disorder are widely used across the world currently, such as the insomnia severity index (ISI), the Berlin questionnaire for sleep apnoea, the questionnaire for snore, tired, observed, pressure (STOP), the International restless legs syndrome rating scale (IRLS), and Epworth sleepiness scale (ESS). These instruments were created for licensed clinicians or sleep specialists to assess and identify the specific disorders and are required to be conducted with supervision. Seven different self-report questionnaires, including Auckland Sleep Questionnaire (ASQ), Global Sleep Assessment Questionnaire (GSAQ), Holland Sleep Disorders Questionnaire (HSDQ), Iowa Sleep Disturbances Inventory (ISDI), Sleep Disorders Questionnaire (SDQ), SLEEP-50, and Sleep Symptom Checklist (SSC) of sleep disorders, reported that to compare with other measurements only GCAQ is feasible to use in the general population at the primary care level, with a suggestion of some further development (Klingman et al., 2017). Moreover, sleep disorders among Korean samples were measured by non-organic and organic sleep disorders according to the International Classification of Diseases, Tenth Edition (ICD-10) (Jin & Choi, 2021). In the study of the general population of Finland, insomnia and sleep habits were measured by the International Classification of Sleep Disorders (ICSD) (Ohayon & Partinen, 2002). Unlike Western countries, the literature on sleep disorders in Asian countries and especially in Mongolia remains sparse.
As sleep disorder is an important risk factor for mental and physical illness, the need for an upgraded assessment of sleep disorder at the primary care level is not merely a critical issue in Mongolia but a global one. Due to the lack of complex and effective screening tools, uncertainty still exists upon primary care providers or general practitioners for assessing sleep disorders and their status among patients. Despite the growing evidence on the importance of screening instruments for sleep disorders, to date, except GCAQ (Roth et al., 2002), there are no appropriate complex questionnaires that screen for multiple sleep disorders for primary care use (Klingman et al., 2017).
Within these existing reasons, this study aimed to develop a specific screening questionnaire on multiple sleep disorders among the general population of Mongolia. This study would be fruitful to individuals suffering from sleep disorders by providing effective screening instruments. On the other hand, it would become a valuable base for policymakers to initiate specific plans for developing national sleep health awareness related to education and awareness of good sleep habits among the general population.
2.0 MATERIALS AND METHODS
2.1 Study design and participants
This study is a test-retest of the questionnaire, in which the respondents answer the same questionnaire on two occasions two weeks apart. For the reliability analysis, the traditional psychometric advises at least 10 participants per scale item (Tinsley & Tinsley, 1987). However, the range from 2 to 20 participants for each item is recently recommended (DeVellis, 2016; Kline, 2016). Since the study was required to be conducted over two time points, the authors decided to collect at least 600 responses in the time 1 (test) of the study to ensure to be recruited an adequate sample size. The medical and non-medical students from the Mongolian National University of Medical Sciences (MNUMC) and the University of Humanities were recruited in this study. A total of 661 participants completed the 30-item draft of the Sleep Disorder Screening Questionnaire (SDSQ) at the time 1 (test). Of these participants, 366 respondents completed the time 2 (retest) questionnaire (Figure 1). Written informed consent was obtained from all participants. All study procedures were carried out in accordance with approval from the Institutional review board and Ethics committee of the Mongolian National University of Medical Sciences (Ethics ID 2021/3-06).
Figure 1: Flowchart of study participants
2.2 Item Generation
Psychiatrist Battuvshin Lhagvasuren, Psychiatrist Gantsetseg Tomor-Ochir, neurobiologist Enkhnaran Tumurbaatar, and Public health professional Enkhzaya Batkhuyag initiated and constructed the sleep disorder screening questionnaire (SDSQ). The questionnaire items were produced via discussion with professional neuroscientists, psychiatrists, and public health professionals. All items in the questionnaire were systematically generated to cover the main sleep disorders classification based on the assessments of ICD-10. ICD-10 has been widely used in Mongolia to investigate sleep disorders. It assesses and classifies the sleep conditions and groups sleep disorders into global categories (Bathgate & Edinger, 2019).
In the current study, the questionnaire consisted of a total of 30 items with contents that covered non-organic insomnia (5 items; Q 1;4;8;11;17), a mental disorder related insomnia (3 items; Q 5;13;30), circadian rhythm related sleep disorder (3 items; Q 9;22;23), restless leg syndrome (2 items; Q 18;26), lucid dream and parasomnia (2 items; Q 19;27), sleep apnoea (2 items; Q 14;28), night terror and nightmare disorder (5 items; Q 7;10;15;24;29), narcolepsy (5 items; Q 2;6;12;16;21), and an assessment of causes of sleep disorder (3 items; Q 3;20;25). The questionnaire assessed the participants’ sleeping experience over the past 6 months. The items except 3, 20 and 25 consisted of statements and a 5-point Likert scale of agreement, with “Never” being the point furthest on the left and “Always” being the furthest point on the right. The following values were assigned to responses: 0= “Never”, 1= “Rarely”, 2= “Sometimes”, 3= “Usually”, 4= “Always”, apart from negatively worded items which were reversely scored.
The specific scoring guide for the SDSQ evaluates contents separately and sums of item responses to designate whether participants need additional investigation from the licensed clinician or sleep specialist. Further, a higher sum score indicating a greater likelihood for the presence of general sleep problems. The suspected score of certain sleep disorders refer to A) Non-organic insomnia with a total score ≥ 8, B) Mental disorder-related insomnia total score ≥ 6, C) Circadian rhythm related sleep disorder with a total score ≥ 6, D) Restless leg syndrome with a total score ≥ 4, E) Lucid dream and parasomnia with total score ≥ 4, F) Sleep apnoea with total score ≥ 4, J) Night terror and nightmare disorder with total score ≥ 8, H) Narcolepsy total score ≥ 8. Yes-no questions assessed the causes of sleep disorders. (Q 3; 20; 25) The answer “yes” leads respondents to further deepened the investigation of the licensed clinician or sleep specialist. Finally, demographic information such as age, gender, living condition, and residence location was included in the questionnaire. The predicted time to complete the current questionnaire was 15 minutes for each participant.
2.4 Procedure
The study was carried out between April and May 2021. Due to the local outbreak of Covid-19, starting from November 2021, all of the educational institutes and universities shifted to cyber-learning. Thus, the current study participants completed the distributed questionnaire online with some general questions to obtain demographic data. A total of 661 participants completed the 30-item draft SDSQ at the time 1 (test). The questionnaire took approximately 15-20 minutes to complete, and the participants answered the same questionnaire on two occasions two weeks apart. Twelve days after the first response, participants were sent a reminder notice via e-mail to the questionnaire for the second occasion. A total of 366 participants completed the time 2 (retest) questionnaire. The research assistants were trained to apply SDSQ, the scoring method, and the assessment of the result.
2.5 Statistical Analysis
Reliability between 2 times (test-retest) of SDSQ was reported by the calculation of intra-class correlations coefficient (ICC) estimates and 95% confidence interval (CI) by using SPSS statistical package version 26 based on a two-way mixed-effects model with the absolute agreement. The results were presented as frequency distribution or mean ± standard deviation range for the demographic characteristics and SDSQ. The ICC values above 0.7 or 0.9 demonstrated good or excellent reliability (Koo & Li, 2016). Compared with Pearson or Spearman correlation coefficients, the ICC considers measurement errors and suitability in test-retest analysis (de Vet et al., 2006). The adequate time interval for test-retest is 10-14 days (Keszei et al., 2010). Moreover, Cronbach’s alpha coefficients were calculated to assess the internal consistency of SDSQ. Cronbach’s Alpha coefficient demonstrates the covariance level between the items of the scale or test, and it has been a widely used measure to assess reliability. The higher the alpha coefficient, the more the items have shared covariance and certainly measure the same content (Tavakol & Dennick, 2011).
3.0 RESULTS
3.1 Demographics
The sample included 366 participants with a mean age of 21.6±1.9 years. Of these participants, 194 (53%) were female. The majority of respondents (55.7%) were studied at the Mongolian National University of Medical Science, and residence locations were in cities (51.9%), followed by western area/provinces (16.7%) mountain area (14.2%). In the current study, 322 (88%) participants were living in an apartment or house, while 8.5% of the respondents were living in a Ger, the traditional portable dwelling (Table 1)
3.2 Frequency distribution of the SDSQ items
Table 2 shows the frequency of item answers in test and retest questionnaires. The majority of participants were reported rarely to sometimes in questions of non-organic insomnia (Q 1;4;8;11;17). Most answers on mental disorder-related insomnia, circadian rhythm related sleep disorder, and restless leg syndrome were given as never-to-rarely. The majority of participants answered never had sleep apnoea, as well as lucid dreams and parasomnia. The most frequent answers on narcolepsy and night terror and nightmare were never and rarely.
Table 1. Demographic characteristics of study participants
Characteristics | Patients | ||
n | % | ||
Age (years), mean ± SD | 21.6±1.9 | ||
Age group |
|
| |
| <19 | 47 | 12.8 |
| 20-24 | 308 | 84.2 |
| >25 | 11 | 3.0 |
Gender |
|
| |
| Male | 172 | 47.0 |
| Female | 194 | 53.0 |
University |
|
| |
| MNUMS | 204 | 55.7 |
| Other University | 162 | 44.3 |
Residence location |
|
| |
| Cities | 190 | 51.9 |
| Eastern | 14 | 3.8 |
| Central | 49 | 13.4 |
| Mountain | 52 | 14.2 |
| Western | 61 | 16.7 |
Living condition |
|
| |
| Apartment | 253 | 69.1 |
| Ger | 31 | 8.5 |
| House | 69 | 18.9 |
| Other | 13 | 3.6 |
Total | 366 | 100.0 | |
Table 2. Frequency distribution of the SDSQ items.
Question
| Never | Ralely | Sometimes | Usually | Always | |||||
Test | Retest | Test | Retest | Test | Retest | Test | Retest | Test | Retest | |
Non-organic insomnia | ||||||||||
Q1 n (%) | 26 (7.1) | 31 (8.5) | 81 (22.1) | 133 (36.3) | 121 (33.1) | 99 (27.0) | 79 (21.6) | 69 (18.9) | 59 (16.1) | 34 (9.3) |
Q4 n (%) | 81 (22.1) | 80 (21.9) | 184 (50.3) | 188 (51.4) | 74 (20.2) | 71 (19.4) | 21 (5.7) | 20 (5.5) | 6 (1.6) | 7 (1.9) |
Q8 n (%) | 32 (8.7) | 50 (13.7) | 143 (39.1) | 140 (38.3) | 134 (36.6) | 124 (33.9) | 47 (12.8) | 39 (10.7) | 10 (2.7) | 13 (3.6) |
Q11 n (%) | 98 (26.8) | 82 (22.4) | 159 (43.4) | 189 (51.6) | 82 (22.4) | 72 (19.7) | 19 (5.2) | 20 (5.5) | 8 (2.2) | 3 (0.8) |
Q17 n (%) | 283 (77.3) | 264 (72.1) | 57 (15.6) | 67 (18.3) | 22 (6.0) | 30 (8.2) | 3 (0.8) | 5 (1.4) | 1 (0.3) | – |
Mental disorder related insomnia | ||||||||||
Q5 n (%) | 92 (25.1) | 82 (22.4) | 117 (32.0) | 138 (37.7) | 120 (32.8) | 102 (27.9) | 34 (9.3) | 32 (8.7) | 3 (0.8) | 12 (3.3) |
Q13 n (%) | 39 (10.7) | 41 (11.2) | 124 (33.9) | 162 (44.3) | 147 (40.2) | 116 (31.7) | 44 (12.0) | 37 (10.1) | 12 (3.3) | 10 (2.7) |
Q30 n (%) | 53 (14.5) | 68 (18.6) | 145 (39.6) | 149 (40.7) | 106 (29.0) | 105 (28.7) | 47 (12.8) | 35 (9.6) | 15 (4.1) | 9 (2.5) |
Circadian rhythm related sleep disorder | ||||||||||
Q9 n (%) | 162 (44.3) | 119 (32.5) | 101 (27.6) | 135 (36.9) | 70 (19.1) | 87 (23.8) | 24 (6.6) | 18 (4.9) | 9 (2.5) | 7 (1.9) |
Q22 n (%) | 73 (19.9) | 71 (19.4) | 142 (38.8) | 164 (44.8) | 101 (27.6) | 94 (25.7) | 40 (10.9) | 27 (7.4) | 10 (2.7) | 10 (2.7) |
Q23 n (%) | 144 (39.3) | 153 (41.8) | 116 (31.7) | 129 (35.2) | 76 (20.8) | 58 (15.8) | 19 (5.2) | 19 (5.2) | 11 (3.0) | 7 (1.9) |
Restless leg syndrome | ||||||||||
Q18 n (%) | 201 (54.9) | 178 (48.6) | 87 (23.8) | 107 (29.2) | 55 (15.0) | 57 (15.6) | 16 (4.4) | 20 (5.5) | 7 (1.9) | 4 (1.1) |
Q26 n (%) | 83 (22.7) | 98 (26.8) | 134 (36.6) | 131 (35.8) | 76 (20.8) | 79 (21.6) | 54 (14.8) | 43 (11.7) | 19 (5.2) | 15 (4.1) |
Lucid dream and parasomnia | ||||||||||
Q19 n (%) | 297 (81.1) | 279 (76.2) | 50 (13.7) | 63 (17.2) | 17 (4.6) | 20 (5.5) | 1 (0.3) | 3 (0.8) | 1 (0.3) | 1 (0.3) |
Q27 n (%) | 178 (48.6) | 183 (50.0) | 92 (25.1) | 100 (27.3) | 57 (15.6) | 55 (15.0) | 27 (7.4) | 23 (6.3) | 12 (3.3) | 5 (1.4) |
Sleep apnoea | ||||||||||
Q14 n (%) | 156 (42.6) | 148 (40.4) | 127 (34.7) | 135 (36.9) | 65 (17.8) | 69 (18.9) | 16 (4.4) | 13 (3.6) | 2 (0.5) | 1 (0.3) |
Q28 n (%) | 305 (83.3) | 289 (79.0) | 39 (10.7) | 49 (13.4) | 14 (3.8) | 26 (7.1) | 6 (1.6) | 1 (0.3) | 2 (0.5) | 1 (0.3) |
Night terror and nightmare disorder | ||||||||||
Q7 n (%) | 62 (16.9) | 76 (20.8) | 198 (54.1) | 193 (52.7) | 78 (21.3) | 72 (19.7) | 19 (5.2) | 18 (4.9) | 9 (2.5) | 7 (1.9) |
Q10 n (%) | 196 (53.6) | 184 (50.3) | 133 (36.3) | 132 (36.1) | 28 (7.7) | 45 (12.3) | 4 (1.1) | 4 (1.1) | 5 (1.4) | 1 (0.3) |
Q15 n (%) | 164 (44.8) | 161 (44.0) | 129 (35.2) | 131 (35.8) | 50 (13.7) | 53 (14.5) | 17 (4.6) | 15 (4.1) | 6 (1.6) | 6 (1.6) |
Q24 n (%) | 250 (68.3) | 242 (66.1) | 88 (24.0) | 87 (23.8) | 19 (5.2) | 33 (9.0) | 7 (1.9) | 3 (0.8) | 2 (0.5) | 1 (0.3) |
Q29 n (%) | 229 (62.6) | 215 (58.7) | 79 (21.6) | 98 (26.8) | 41 (11.2) | 34 (9.3) | 12 (3.3) | 13 (3.6) | 5 (1.4) | 6 (1.6) |
Narcolepsy | ||||||||||
Q2 n (%) | 244 (66.7) | 226 (61.7) | 78 (21.3) | 83 (22.7) | 38 (10.4) | 49 (13.4) | 5 (1.4) | 7 (1.9) | 1 (0.3) | 1 (0.3) |
Q6 n (%) | 128 (35.0) | 116 (31.7) | 153 (41.8) | 159 (43.4) | 71 (19.4) | 70 (19.1) | 12 (3.3) | 19 (5.2) | 2 (0.5) | 2 (0.5) |
Q12 n (%) | 69 (18.9) | 65 (17.8) | 136 (37.2) | 159 (43.4) | 115 (31.4) | 105 (28.7) | 37 (10.1) | 32 (8.7) | 9 (2.5) | 5 (1.4) |
Q16 n (%) | 104 (28.4) | 94 (25.7) | 145 (39.6) | 161 (44.0) | 89 (24.3) | 83 (22.7) | 22 (6.0) | 20 (5.5) | 6 (1.6) | 8 (2.2) |
Q21 n (%) | 56 (15.3) | 64 (17.5) | 188 (51.4) | 172 (47.0) | 92 (25.1) | 97 (26.5) | 24 (6.6) | 24 (6.6) | 6 (1.6) | 9 (2.5) |
|
| The causes of sleep problem |
| |||||||
Question | Answer |
|
|
| ||||||
No |
| Yes |
|
|
| |||||
n | % |
| n | % |
|
|
| |||
Q3 n (%) |
|
| 365 | 99.7 |
| 1 | 0.3 |
|
|
|
Q20 n (%) |
|
| 359 | 98.1 |
| 7 | 1.9 |
|
|
|
Q25 n (%) |
|
| 189 | 51.6 |
| 177 | 48.4 |
|
|
|
3.3 Test-retest reliability
To determine the test-retest reliability of the SDSQ, an ICC and Cronbach’s alpha calculation was employed on a total of 366 participants. Table 3 shows the result of the reliability of the test and retest. The current study employed a two-way mixed-effects model with absolute agreement definition. The correlation between scores from test and retest was good (ICC ≥ 0.700) with 95% CI for 21 items (Q 1,3-8,10,13-18,20-22,25-27,29-30) and moderate (ICC= 0.530-0.690) for remained 9 items. To investigate the internal consistency of the questionnaire, the Cronbach’s alphas were performed on both test and retest questionnaires and revealed a good level of internal consistency (alpha =0.950). Deletion of any item did not increase or decrease the alpha by more than 0.01. Together these results demonstrating that the questions of the SDSQ were consistent at two times of screening that measured sleep disorders among study participants (Table 3). These observations also are supported by the linear regression and the scatterplot (regression coefficient, 0.505) of total scores of test and retest (Figure 2). The result in Figure 2 indicates a high relationship between the test and retest, showing the strong reliability of data collected on these two occasions.
Table 3. Correlation and homogeneity of the SDSQ.
Question | Intraclass correlation | 95% Confidence Interval | F Value | P value | Cronbach’s Alpha* if Item Deleted | ||
Lower Bound | Upper Bound | Test | Retest | ||||
Q1 | 0.761 | 0.706 | 0.805 | 4.183 | 0.000 | 0.949 | 0.949 |
Q2 | 0.670 | 0.595 | 0.731 | 3.029 | 0.000 | 0.950 | 0.950 |
Q3 | 0.565 | 0.466 | 0.646 | 2.298 | 0.000 | 0.950 | 0.950 |
Q4 | 0.713 | 0.648 | 0.766 | 3.486 | 0.000 | 0.949 | 0.949 |
Q5 | 0.730 | 0.669 | 0.780 | 3.705 | 0.000 | 0.949 | 0.949 |
Q6 | 0.756 | 0.700 | 0.801 | 4.092 | 0.000 | 0.949 | 0.949 |
Q7 | 0.712 | 0.646 | 0.766 | 3.474 | 0.000 | 0.949 | 0.949 |
Q8 | 0.758 | 0.703 | 0.803 | 4.139 | 0.000 | 0.949 | 0.949 |
Q9 | 0.648 | 0.568 | 0.713 | 2.841 | 0.000 | 0.949 | 0.949 |
Q10 | 0.714 | 0.649 | 0.767 | 3.498 | 0.000 | 0.949 | 0.949 |
Q11 | 0.536 | 0.431 | 0.622 | 2.157 | 0.000 | 0.951 | 0.950 |
Q12 | 0.682 | 0.609 | 0.741 | 3.142 | 0.000 | 0.950 | 0.950 |
Q13 | 0.733 | 0.673 | 0.783 | 3.752 | 0.000 | 0.949 | 0.949 |
Q14 | 0.805 | 0.760 | 0.841 | 5.125 | 0.000 | 0.950 | 0.950 |
Q15 | 0.732 | 0.671 | 0.782 | 3.733 | 0.000 | 0.949 | 0.949 |
Q16 | 0.741 | 0.681 | 0.789 | 3.855 | 0.000 | 0.949 | 0.949 |
Q17 | 0.778 | 0.727 | 0.819 | 4.496 | 0.000 | 0.950 | 0.949 |
Q18 | 0.775 | 0.723 | 0.817 | 4.439 | 0.000 | 0.949 | 0.949 |
Q19 | 0.691 | 0.620 | 0.748 | 3.236 | 0.000 | 0.950 | 0.950 |
Q20 | 0.745 | 0.687 | 0.793 | 3.928 | 0.000 | 0.950 | 0.950 |
Q21 | 0.751 | 0.694 | 0.797 | 4.008 | 0.000 | 0.949 | 0.948 |
Q22 | 0.712 | 0.647 | 0.766 | 3.477 | 0.000 | 0.949 | 0.948 |
Q23 | 0.619 | 0.532 | 0.690 | 2.627 | 0.000 | 0.950 | 0.950 |
Q24 | 0.564 | 0.465 | 0.645 | 2.295 | 0.000 | 0.950 | 0.950 |
Q25 | 0.718 | 0.654 | 0.770 | 3.546 | 0.000 | 0.950 | 0.949 |
Q26 | 0.740 | 0.681 | 0.789 | 3.853 | 0.000 | 0.949 | 0.949 |
Q27 | 0.809 | 0.765 | 0.844 | 5.228 | 0.000 | 0.951 | 0.950 |
Q28 | 0.629 | 0.544 | 0.698 | 2.693 | 0.000 | 0.950 | 0.950 |
Q29 | 0.708 | 0.642 | 0.763 | 3.430 | 0.000 | 0.950 | 0.949 |
Q30 | 0.723 | 0.660 | 0.775 | 3.614 | 0.000 | 0.949 | 0.949 |
Figure 2: Scatter plots of linear regression model comparing the total score of SDSQ test and retest. Black lines are linear least squares regression fits to data points.
4.0 DISCUSSION
The purpose of the study was to determine the test-retest reliability of a screening instrument for multiple sleep disorders. The study finding demonstrated the excellent level of covariance between the items of the current instrument (Cronbach’s alpha = 0.950). In other words, this result indicating that SDSQ is consistent at measuring the same content, and this questionnaire is feasible to employ in general practice. This finding was consistent with reports from previous studies of similar screening tools for the sleep disorder. The internal reliability coefficient alpha of the Holland Sleep Disorders Questionnaire (HSDQ) was 0.90 (Kerkhof et al., 2013). The Iowa Sleep Disturbances Inventory (ISDI) study reported Cronbach’s alphas as 0.80 or greater (Koffel & Watson, 2010). A similar result was observed in the study of the Sleep-50 questionnaire as Cronbach’s alpha=0.85 (Spoormaker et al., 2005). The current finding of the study may have the following reasons: (1) the item generation and the instrument development were carried out from real clinical practice; (2) the instrument was created and generated by Mongolian neuroscientists and licensed clinicians in the Mongolian language, thus, it was understandable and accessible to the study participants.
The intra-class correlation of the SDSQ was considered satisfactory. It can be interpreted as two times of SDSQ measured same content over time, suggesting that SDSQ was a reliable instrument in this study sample. The screening questionnaires for single specific sleep disorders reported similar results previously. For instance, the study revealed the reliability of the Arabic version of STOP-BANG as ICC=0960 (BaHammam et al., 2015). The test-retest reliability of the Thai version of the Epworth Sleepiness Scale was ICC=0.760 (Banhiran et al., 2011) and among the Turkish version was ICC = 0.81 (Izci et al., 2008), while in the Arabic version was ICC=0.860 (Ahmed et al., 2014). As earlier mentioned in the introduction part, to our knowledge, at the primary care level, except GSAQ, there was no complex screening questionnaire that is available to detect multiple sleep disorders. (Klingman et al., 2017). A similar to our finding, the study on GSAQ test-retest reliability reported the ICC ranged from 0.51 to 0.92 (Roth et al., 2002). By developing the SDSQ, we aimed to assess sleep quality and detect multiple sleep disorders among the general population. Although this was the first attempt to construct a new instrument, strong correlations were found upon most items (21 out of the 30) of test and retest questionnaires. This result was also confirmed by the result of linear regression of the total scores of the test and retest questionnaire. Moderate correlations were observed for some items (9 out of 30) of the test and retest questionnaire. For instance, some narcolepsy and circadian rhythm-related sleep disorders were observed to be moderately correlated in test-retest (ICC ≤ 0.700). The circadian rhythm-related sleep disorder is well detected by the overnight study as evaluation with a diagnostic tool (Ibáñez et al., 2018), thus, it was maybe hard to identify such disorders only use the questionnaire.
Moreover, the ICC in the responses among item 11 of non-organic insomnia was at a moderate level. In this item, we questioned participants as “Do you wake up earlier than your normal schedule? (30 min or more)”. We assume that the study participants might pay increased attention to the word “30 minutes” but not to “more”. For this reason, in the following study, we should consider the construct of our questions. Two more items in narcolepsy and one item in parasomnia and lucid dream were moderately correlated in test and retest. Due to the relatively small sample size, the study of GSAQ reported ICC in the responses of parasomnia was 0.51. In our study, we had recruited an adequate sample size according to the requirement of study of the reliability of test-retest (DeVellis, 2016, Kline 2016). hence a small number of items on these contents might be a reason for this result. These results are together suggesting the need to conduct factor analysis of the construct of SDSQ further.
A possible limitation could be addressed in this study. Since this is a self-report questionnaire, it is difficult to prove that participants correctly assessed their sleeping experience. The possibility of the participants providing insufficient information may exist. Due to the local outbreak of Covid-19, several times quarantine that occupied in Mongolia followed with sudden changes in study conditions (Erkhembayar et al., 2020), the sleeping experience among students may have been different from the ordinary circumstances. Although the self-report questionnaires may have some bias because of participants’ feelings or the attitude at the time they filled out the questionnaire, it is worthy to note that the advantage of allowing participants to describe their own experiences without any interference (Demetriou et al., 2015).
Within the purpose of improvement of the questionnaire, further research is suggested to conduct of the SDSQ study in both general and clinical population; with a suggestion to include participants’ mental health assessment such as anxiety and depression, and vital signs (blood pressure, heart rate, body temperature, respiration rate, BMI, height, and weight) of the study population. The deepened investigation of the construct of the questionnaire should be considered. The strength of this study was that it reflects the actual clinical practice in the Mongolian population. This was an initial attempt to construct and develop specific screening instruments for sleep disorders among Mongolians. Despite the need for further improvement, the SDSQ was an easily administered, simple, reliable instrument for screening sleep disorders in the current study population.
5.0 CONCLUSIONS
The external validity of SDSQ revealed strong test-retest reliability in the current study population. The discriminant and convergent validity studies are required for the further improvement of the construct of the SDSQ.
Acknowledgments
The authors would like to acknowledge the support and contribution of the study team of the Mongolian Brain Science Institute. We greatly appreciate those who participated in our study
Author Contributions
GT, EB, ET, and BL conceived and designed the experiments; EB, GT and EP performed the experiments; GT, ET, MD, EJ, OD contributed /materials/analysis tools; GT, EB, ET, and XZ wrote the paper.
Conflicts of Interest
The authors declare no conflict of interest.
References
Ahmed, A. E., Fatani, A., Al-Harbi, A., Al-Shimemeri, A., Ali, Y. Z., Baharoon, S., & Al-Jahdali, H. (2014). Validation of the Arabic version of the Epworth Sleepiness Scale. Journal of Epidemiology and Global Health, 4(4), 297–302. https://doi.org/10.1016/j.jegh.2014.04.004
BaHammam, A. S., Al-Aqeel, A. M., Alhedyani, A. A., Al-Obaid, G. I., Al-Owais, M. M., & Olaish, A. H. (2015). The Validity and Reliability of an Arabic Version of the STOP-Bang Questionnaire for Identifying Obstructive Sleep Apnea. The Open Respiratory Medicine Journal, 9(1), 22–29. https://doi.org/10.2174/1874306401509010022
Banhiran, W., Assanasen, P., Nopmaneejumruslers, C., & Metheetrairut, C. (2011). Epworth sleepiness scale in obstructive sleep disordered breathing: The reliability and validity of the Thai version. Sleep and Breathing, 15(3), 571–577. https://doi.org/10.1007/s11325-010-0405-9
Bathgate, C. J., & Edinger, J. D. (2019). Chapter 1 – Diagnostic Criteria and Assessment of Sleep Disorders (J. Savard & M.-C. B. T.-H. of S. D. in M. C. Ouellet (eds.); pp. 3–25). Academic Press. https://doi.org/https://doi.org/10.1016/B978-0-12-813014-8.00001-9
Cao, X. L., Wang, S. Bin, Zhong, B. L., Zhang, L., Ungvari, G. S., Ng, C. H., Li, L., Chiu, H. F. K., Lok, G. K. I., Lu, J. P., Jia, F. J., & Xiang, Y. T. (2017). The prevalence of insomnia in the general population in China: A meta-analysis. PLoS ONE, 12(2), 1–11. https://doi.org/10.1371/journal.pone.0170772
de Vet, H. C. W., Terwee, C. B., Knol, D. L., & Bouter, L. M. (2006). When to use agreement versus reliability measures. Journal of Clinical Epidemiology, 59(10), 1033–1039. https://doi.org/10.1016/j.jclinepi.2005.10.015
Demetriou, C., Ozer, B. U., & Essau, C. A. (2015). Self-Report Questionnaires. The Encyclopedia of Clinical Psychology, November 2017, 1–6. https://doi.org/10.1002/9781118625392.wbecp507
Demir, A. U., Ardic, S., Firat, H., Karadeniz, D., Aksu, M., Ucar, Z. Z., Sevim, S., Ozgen, F., Yilmaz, H., Itil, O., Peker, Y., Aygul, F., Kiran, S., Gelbal, S., Cepni, Z., Akozer, M., Neyal, A., Cilli, A., Ozsancak, A., … Bulbul, Y. (2015). Prevalence of sleep disorders in the Turkish adult population epidemiology of sleep study. Sleep and Biological Rhythms, 13(4), 298–308. https://doi.org/10.1111/sbr.12118
DeVellis, R. F. (2016). Scale Development Theory and Applications (Fourth Edition). In SAGE Publication (Vol. 4).
Erkhembayar, R., Dickinson, E., Badarch, D., Narula, I., Warburton, D., Thomas, G. N., Ochir, C., & Manaseki-Holland, S. (2020). Early policy actions and emergency response to the COVID-19 pandemic in Mongolia: experiences and challenges. The Lancet Global Health, 8(9), e1234–e1241. https://doi.org/10.1016/S2214-109X(20)30295-3
Ibáñez, V., Silva, J., & Cauli, O. (2018). A survey on sleep assessment methods. PeerJ, 2018(5), 1–26. https://doi.org/10.7717/peerj.4849
Izci, B., Ardic, S., Firat, H., Sahin, A., Altinors, M., & Karacan, I. (2008). Reliability and validity studies of the Turkish version of the Epworth Sleepiness Scale. Sleep and Breathing, 12(2), 161–168. https://doi.org/10.1007/s11325-007-0145-7
Jin, D., & Choi, J. W. (2021). Sleep disorders and risk of dementia in patients with new-onset type 2 diabetes: A nationwide population-based cohort study. Journal of Diabetes, 13(2), 101–110. https://doi.org/10.1111/1753-0407.13092
Kerkhof, G. A. (2017). Epidemiology of sleep and sleep disorders in The Netherlands. Sleep Medicine, 30, 229–239. https://doi.org/10.1016/j.sleep.2016.09.015
Kerkhof, G. A., Geuke, M. E. H., Brouwer, A., Rijsman, R. M., Schimsheimer, R. J., & Van Kasteel, V. (2013). Holland Sleep Disorders Questionnaire: A new sleep disorders questionnaire based on the International Classification of Sleep Disorders-2. Journal of Sleep Research, 22(1), 104–107. https://doi.org/10.1111/j.1365-2869.2012.01041.x
Keszei, A. P., Novak, M., & Streiner, D. L. (2010). Introduction to health measurement scales. Journal of Psychosomatic Research, 68(4), 319–323. https://doi.org/10.1016/j.jpsychores.2010.01.006
Khaled, S. M., Petcu, C., Al-Thani, M. A., Al-Hamadi, A. M. H. A., Daher-Nashif, S., Zolezzi, M., & Woodruff, P. (2021). Prevalence and associated factors of DSM-5 insomnia disorder in the general population of Qatar. BMC Psychiatry, 21(1), 1–10. https://doi.org/10.1186/s12888-020-03035-8
Kline, R. B. (2016). Principles and practice of structural equation modeling, 4th ed. In Principles and practice of structural equation modeling, 4th ed. Guilford Press.
Klingman, K. J., Jungquist, C. R., & Perlis, M. L. (2017). Questionnaires that screen for multiple sleep disorders. Sleep Medicine Reviews, 32(585), 37–44. https://doi.org/10.1016/j.smrv.2016.02.004
Koffel, E., & Watson, D. (2010). Development and initial validation of the iowa sleep disturbances inventory. Assessment, 17(4), 423–439. https://doi.org/10.1177/1073191110362864
Koo, T. K., & Li, M. Y. (2016). A Guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Li, L., Wang, Y. Y., Wang, S. Bin, Zhang, L., Li, L., Xu, D. D., Ng, C. H., Ungvari, G. S., Cui, X., Liu, Z. M., De Li, S., Jia, F. J., & Xiang, Y. T. (2018). Prevalence of sleep disturbances in Chinese university students: a comprehensive meta-analysis. Journal of Sleep Research, 27(3), 1–13. https://doi.org/10.1111/jsr.12648
Muscogiuri, G., Barrea, L., Annunziata, G., Di Somma, C., Laudisio, D., Colao, A., & Savastano, S. (2019). Obesity and sleep disturbance: the chicken or the egg? Critical Reviews in Food Science and Nutrition, 59(13), 2158–2165. https://doi.org/10.1080/10408398.2018.1506979
National Institutes of Health (2011). National Institutes of Health Sleep Disorders Research Plan. In US Department of Health and Human Services, National Heart Lung and Blood Institute, National Center on Sleep Disorders Research, National Institutes of Health (Issue November).
Ohayon, M. M. (1997). Prevalence of DSM-IV diagnostic criteria of insomnia: Distinguishing insomnia related to mental disorders from sleep disorders. Journal of Psychiatric Research, 31(3), 333–346. https://doi.org/10.1016/S0022-3956(97)00002-2
Ohayon, M. M., & Partinen, M. (2002). Insomnia and global sleep dissatisfaction in Finland. Journal of Sleep Research, 11(4), 339–346. https://doi.org/10.1046/j.1365-2869.2002.00317.x
Roth, T., Zammit, G., Kushida, C., Doghramji, K., Mathias, S. D., Wong, J. M., & Buysse, D. J. (2002). A new questionnaire to detect sleep disorders. Sleep Medicine, 3(2), 99–108. https://doi.org/10.1016/S1389-9457(01)00131-9
Sleep Health Foundation. (2021). Rise and try to shine: The social and economic cost of sleep disorders in Australia Sleep Health Foundation (Issue April).
Spoormaker, V. I., Verbeek, I., van den Bout, J., & Klip, E. C. (2005). Initial validation of the SLEEP-50 questionnaire. Behavioral Sleep Medicine, 3(4), 227–246. https://doi.org/10.1207/s15402010bsm0304_4
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
Tinsley, H. E. A., & Tinsley, D. J. (1987). Uses of factor analysis in counseling psychology research. Journal of Counseling Psychology, 34(4), 414–424. https://doi.org/10.1037/0022-0167.34.4.414
Vela-Bueno, A., De Iceta, M., & Fernández, C. (1999). Prevalence of sleep disorders in Madrid, Spain. Gaceta Sanitaria / S.E.S.P.A.S, 13(6), 441–448. https://doi.org/10.1016/S0213-9111(99)71404-4
Help us to sustain this free open access journal via a voluntary contribution today.
Your contribution will make sure that reading, submitting articles, publishing and indexing of all scientific materials in this journal remain free of charge for everyone.