The reliability and validity of the European League Against Rheumatism Sjögren Syndrome Patient Reported Index in patients with primary Sjögren syndrome: A Turkish version study

Abstract

Objectives: This study aims to assess the reliability and validity of the Turkish version of the European League Against Rheumatism (EULAR) Sjögren Syndrome Patient Reported Index (ESSPRI) (TR) in patients with primary Sjögren syndrome (pSS).

Materials and methods: A cross-sectional survey study design and analysis were used to assess the reliability and validity of the ESSPRI (TR) between March 2019 and July 2019. A total of 30 patients (5 males, 25 females; mean age 54.1±10.5 years; range, 18 to 75 years) diagnosed as pSS according to revised American College of Rheumatology (ACR)/EULAR classification criteria were included. ESSPRI (TR) was applied to the patients with face-to-face interviews twice: on their first visit and after an interval of 15 days. The test-retest reliability was assessed with the intraclass correlation coefficient (ICC), and the internal consistency of the multi-item subscales by calculating Cronbach alpha values. The correlations between basal and stimulated salivary flow (BSF and SSF), Oral Health Impact Profile-14 (OHIP-14) and Oral Health-Related Quality of Life-UK (OHRQOL-UK) questionnaires were evaluated to determine the construct validity.

Results: The ICC value for the test/retest reliability of ESSPRI (TR) was 0.925. The internal consistency was 0.682. There were low to moderate correlations between the ESSPRI (TR) total score and BSF (-0.39), SSF (-0.50), OHIP-14 total (0.57) and OHRQOL-UK total (-0.67).

Conclusion: The Turkish version of the ESSPRI was found to be clinically valid and reliable to be used in clinical evaluations and rehabilitation interventions in patients with pSS.

Introduction

Sjögren syndrome (SS) is a systemic inflammatory disease resulting in the development of dysfunction due to immune damage to the exocrine glands, particularly the salivary and lacrimal glands.[1,2]

The prevalence of SS is approximately 0.5-5%. Although SS can occur at all ages, it is more common in females aged 40-50 years,[3] and nine times more common in females than in males.[4] Up to 50% of patients are primary Sjögren’s syndrome (pSS) cases.[5] pSS prevalence in Turkey has been found to be 0.72% according to the American-European criteria and to be 1.56% according to the European criteria.[6] The clinical picture and serological findings do not differ greatly in terms of age, sex and ethnicity in different societies.[6,7]

Patients with pSS often complain of dry mouth (xerostomy).[8] Dry mouth may be an initial symptom.[9] Due to lack of saliva, there is a burning sensation extending to the throat, which makes it difficult to talk and particularly to chew and swallow dry foods. Patients complain of increased viscosity saliva, pain and a burning sensation in the tongue, and taste disorders. Fluid intake is increased. Tooth decay is accelerated due to decreased saliva and loss of the antibacterial effects of saliva and changes in oral flora and pH. Hoarseness and dry cough may also occur due to problems in the oropharynx and upper respiratory tract. Oral examination of the patients shows dryness, hyperemia, and a thin and parchmentlike appearance of the oral mucosa. Dry lips and cracks in the lip corners and oral candidiasis may be noticed. Unilateral or bilateral swelling of the parotid glands occurs in 60% of patients with pSS.[10-12]

Although pSS is a common disease with an impact on patients, there is little evidence for treatment. Evidence-based evaluation of treatments is important in planning clinical trials, and major efforts are being made to determine outcome measures in pSS.[13]

Different indexes have been developed for the evaluation of disease-related symptoms.[14-16] However, these indexes generally evaluate a single aspect of the disease. For example, the Profile of Fatigue and Discomfort (PROFAD) and the Sicca Symptoms Inventory (SSI) assess fatigue and sicca symptoms, and the Sjögren syndrome Disease Activity Index and Sjögren’s Systemic Clinical Activity Index evaluate the patient’s systemic status. The scoring of all these indexes is detailed and time consuming.[14-17] Therefore, the European League Against Rheumatism (EULAR) developed the EULAR Sjögren Syndrome Patient Reported Index (ESSPRI) following a multinational study to assess the perception of the most common symptoms by the patient, and it has been used as an outcome measure in clinical trials. ESSPRI has been shown to correlate with the commonly used quality of life and functional status measures.[18]

As a result, it is used as the gold standard in the overall evaluation of the patient's self-reported disease activity. ESSPRI is a symptom severity scale to evaluate SS and completed by the patient. The patient is asked to express complaints of fatigue, pain and dryness with a score of 1 to 10. The total score is the result of the arithmetic mean of the scores obtained from these three questions. While an ESSPRI score of <5 is considered as acceptable disease status, a score of ≥5 indicates high disease activity finding.[18]

Other than in English, the index has proven validity and reliability in the Brazilian Portuguese language.[19] Validity and reliability studies of ESSPRI have not been conducted in Turkish. Therefore, in this study, we aimed to assess the reliability and validity of the Turkish version of ESSPRI (TR) in patients with pSS.

Patients and Methods

This cross-sectional survey study was conducted at Department of Rheumatology, Medical Faculty of Pamukkale University between March 2019 and July 2019. The study sample consisted of 30 patients (5 males, 25 females; mean age 54.1±10.5 years; range, 18 to 75 years) diagnosed with pSS according to the revised American College of Rheumatology (ACR)/EULAR classification criteria. It has been reported in the literature that the number of samples should be at least five times the number of items20 and the incidence of the disease is low.[21]

The inclusion criteria were (i) a diagnosis of pSS according to the revised ACR/EULAR classification criteria, (ii) voluntary participation in the study, (iii) age >18 years, and (iv) regular use of rheumatic drugs for at least three months before the study.

The exclusion criteria were (i) the presence of another autoimmune disease, (ii) cirrhosis, sarcoidosis, or known hepatitis infection, (iii) acquired immunodeficiency syndrome, (iv) pre-existing lymphoma, (v) graft versus disease, (vi) a history of radiation to the head and neck region, or (vii) a history of anticholinergic drug use. The data of any participant whose drug treatment was changed during the study were not included and the participation of the participant was terminated. The study protocol was approved by the Medical Faculty of Pamukkale University Ethics Committee (meeting no. 15, dated: 10.09.2019). A written informed consent was obtained from each patient. The study was conducted in accordance with the principles of the Declaration of Helsinki.

Demographic data were collected via a structured interview. All participants were evaluated in the outpatient clinic by the same investigator. Oral health-related quality of life was assessed using the Oral Health Impact Profile-14 (OHIP-14) and Oral Health-Related Quality of Life United Kingdom (OHRQOL-UK) questionnaires. Then, after 5 minutes of rest in a quiet laboratory setting, the basal and stimulated (2% citric acid) salivary flow (BSF-SSF) rate was measured using the salivary flow rate test. The evaluation was performed in a single session lasting 45-60 minutes.

During the ESSPRI cross-cultural adaptation process, previously recommended procedures were followed in five stages.[22,23] First, the questionnaire was translated from English to Turkish by two different independent translators, whose native language was Turkish, and then both translations were synthesized into one. Second, another two independent translators (native English speakers), who were unaware of the original items, performed back-translations of the previously obtained Turkish translation of the questionnaire. After the translations were completed, four physiotherapists experienced in pSS, who did not participate in the translation processes, held a consensus meeting and prepared the pre-final version. Finally, the pre-final version of ESSPRI (TR) was administered to 10 patients with pSS to evaluate comprehensibility and to determine other linguistic fine-tuning. During these interviews, no conflicts were obtained related to the comprehension or clarity of the items. The three-item ESSPRI (TR) is presented in Appendix 1.

The ESSPRI is a symptom severity assessment questionnaire completed by the patient and used to evaluate SS. The patient is asked to express complaints of fatigue, pain and dryness with a score of 0-10. While a score of 0 indicates no symptoms, a score of 10 indicates the highest level of symptoms. When scoring, the patient is asked to mark the scoring that best describes his or her condition in the last two weeks. The patient should indicate the option that best describes the severity of his or her symptoms over the past two weeks. Each area independently indicates the severity of the symptom. The final score is obtained by averaging the scores of the three domains. While an ESSPRI score of <5 is considered as acceptable disease status, a score of ≥5 indicates high disease activity findings.[18]

The OHIP, a validated and reliable instrument, was used to assess the quality of life associated with oral health. OHIP-14 consists of 14 questions and five optional responses for each question, scored between 0-4 as a Likert scale: “0”=never, “1”=rarely, “2”=sometimes, “3”=often and “4”=always. OHIP-14 consists of seven dimensions: functional limitation (questions 1 and 2), physical pain (questions 3 and 4), psychological discomfort (questions 5 and 6), physical disability (questions 7 and 8), psychological disability (questions 9 and 10), social handicap (questions 11 and 12) and handicap (questions 13 and 14). Dimensions can be scored individually and the total OHIP score is obtained as the total of the scores of these seven dimensions. High scores indicate that quality of life is adversely affected.[24] The reliability and validity studies of the Turkish version of OHIP-14 were conducted in 2006 by Mumcu et al.[25]

The OHRQOL-UK was developed in 2000 by McGrath and Bedi in England and it consists of 16 questions assessing the impact of the individual's orofacial status on their quality of life. OHRQOL-UK consists of four dimensions: symptoms (questions 1 and 2), physical status (questions 3, 4, 5, 6 and 7), psychological status (questions 8, 9, 10, 11 and 12) and social status (questions 13, 14, 15 and 16). These questions are scored on a five-point Likert-type scale with the total score ranging from 16 to 80. Low scores indicate that the quality of life related to oral-dental health is low.[26] The reliability and validity studies of the Turkish version of OHRQOL-UK were conducted in 2006 by Mumcu et al.[25]

For the salivary flow rate test measurement, patients with pSS were asked to come to the clinic at 09:00 without eating or drinking anything in the morning. To determine the amount of basal saliva, the individual was rested for 5 min before starting the measurement. Then, the weight of the measuring cup was determined and the individual was asked to collect his or her saliva into the measuring cup for 10 minutes in a quiet environment. The amount of saliva collected was measured on a sensitive scale and recorded in grams. The method for measuring the amount of stimulated saliva was the same. Unlike the method of measuring the amount of basal saliva, two drops of citric acid (2%) were placed on the individual's tongue to stimulate the amount of saliva just before starting the measurement of the amount of stimulated saliva.[27]

Table 1. Demographic and disease-related information of individuals included in study

The distribution of continuous variables was examined with the Shapiro-Wilk test. In the representation of descriptive statistics for numerical variables, mean±standard deviation and median (minimum-maximum) values were stated. Number and percentage were used for categorical variables. Construct validity was examined using factor analysis. Factor loadings and Bartlett's test of sphericity were calculated. External construct validity (convergent validity) was evaluated with the Spearman correlation coefficient. The internal consistency reliability of the scale was examined using the Cronbach alpha coefficient. Reproducibility (test-retest reliability) was assessed with the intraclass correlation coefficient (ICC, two-way mixed effects, absolute agreement).[28] A Bland-Altman plot was used to evaluate the agreement between test and retest. The one sample t-test was applied to determine whether the mean of the difference was different from zero. For all statistical analyses, IBM SPSS Statistics version 21.0 software was used (IBM Corp, Armonk, NY, USA). A value of p<0.05 was accepted as statistically significant.

Table 2. Descriptive statistics of applied scales and clinical findings

Table 3. Descriptive statistics for each question and total score of ESSPRI

Results

The demographic and disease-related information of the patients included in the study is given in Table 1. The results of the salivary flow rate test and scores of OHIP-14 and OHRQOL-UK questionnaires are summarized in Table 2.

As a result of the Bartlett test, the correlation matrix was found to be suitable for factor analysis (Bartlett’s c2=13.014, df=3, p<0.005). When the factor structure of the ESSPRI (TR) was examined, it was found that three items were combined in a single factor and the variance explanation rate was 61.31%. Factor loadings were obtained as 0.780, 0.785 and 0.784, respectively. The total score of ESSPRI (TR) and descriptors for each item are given in Table 3.

Table 4 shows the correlation between the ESSPRI (TR) and OHIP-14 and OHRQOL-UK scores. There was a significant positive correlation between ESPRI (TR) total score and the total score of OHIP-14 (Figure 1a) and the subdimension scores of OHIP-14 (p<0.05). A significant negative correlation was determined between the ESSPRI (TR) total score and the total score of OHRQOL-UK (Figure 1b) and the physical status dimension, psychological status dimension and social status dimension scores of OHRQOL-UK (p<0.05). The correlation between the ESSPRI (TR) and the symptom dimension of OHRQOL-UK was not significant (p=0.075).

Table 4. Correlation between ESSPRI and OHIP-14 and OHRQOL-UK questionnaires

The test-retest reliability of ESSPRI (TR) was found to be quite high. The obtained intraclass correlation coefficients and confidence intervals were given in Table 5. When the difference between the test and retest was examined with the Bland-Altman plot, there was no value exceeding the confidence limit for the difference and the differences were distributed randomly around zero (p=0.186; Figure 2).

Figure 1. Scatter plots of European League Against Rheumatism Sjögren Syndrome Patient Reported Index and Oral Health Impact Profile-14 (a) and Oral Health-Related Quality of Life-UK (b) questionnaires. ESSPRI: European League Against Rheumatism Sjögren Syndrome Patient Reported Index; OHIP-14: Oral Health Impact Profile-14; OHRQOL-UK: Oral Health-Related Quality of Life-UK; rho: Spearman correlation coefficient.

Table 5. Test-retest reliability of ESSPRI

Figure 2. Plot of difference scores against average of European League Against Rheumatism Sjögren Syndrome Patient Reported Index 1 and 2 measurements. ESSPRI: European League Against Rheumatism Sjögren Syndrome Patient Reported Index; SD: Standard deviation.

Discussion

The Turkish version of the ESSPRI was found to be clinically valid and reliable for use in clinical evaluations, and rehabilitation interventions in patients with pSS.

The ESSPRI was created with the participation of a multinational patient panel from Europe, and North and South American countries. The aim was to include the opinions of patients in different cultures regarding the importance and severity of symptoms.[18] The score obtained from the patient's self-assessment was of benefit in demonstrating that ESSPRI was easy to use and satisfactory in the assessment of patient symptoms in pSS.15,16 Although ESSPRI is a subjective assessment method for the severity of pSS symptoms, it is currently the most widely used system internationally, with the advantage of responding more quickly to the patient's perception.[18]

Prior to the development of ESSPRI, SSI was used to assess the characteristics of symptoms of dryness, and PROFAD was used for fatigue and discomfort symptoms, but these assessment scales did not have a global score.[15,16] Thus, ESSPRI provides a significant advantage as both a general score while separate scores for sub-items are obtained which can evaluate all symptoms of the disease. As the index has a global score, it is also a consistent aid in designing and conducting clinical trials in pSS.[18]

The ESSPRI was initially composed of 10 items to question general symptoms such as fatigue, pain and mental fatigue, and dryness in six areas (eye, mouth, vagina, skin, nose, airway and respiration). In an international study, the psychometric properties of the threequestion index were found to be as good as the 10-question index. Therefore, the index is limited to three global questions.[18]

Test-retest reliability measures the stability of answers to questions of a scale over time. After the first and second evaluation of the ESSPRI (TR), the ICC score was 0.925 for the total score, 0.950 for dryness, 0.904 for fatigue and 0.835 for pain. With these results, test-retest reliability was obtained at excellent and good levels. With a Bland-Altman plot (Figure 1), there was no systematic bias in understanding the measurements between the test and retest, and the data were well distributed. Likewise, internal consistency is important to measure a basic construct in multiple items. The internal consistency coefficient (Cronbach alpha) for ESSPRI (TR) was determined as 0.682, showing that the scale was very reliable. The Brazilian Portuguese study of ESSPRI is the only other language version study in the literature. In that study, the ICC score was 0.911 and test-retest reliability was reported to be excellent.[19] However, unlike the current study, the Brazilian Portuguese version reported that the internal consistency coefficient was 0.447 and weak. The Turkish version results show that ESSPRI (TR) is reliable and valid.

Since BSF-SSF rate is an objective evaluation method, and OHIP-14 and OHRQOL-UK questionnaires, examining the effectiveness of oral health on quality of life, are widely accepted because of their reliability and validity, they were determined as the gold standard for determining the construct validity.[24-26] Therefore, these assessment methods were used to evaluate the construct validity of ESSPRI (TR). A weak and moderate correlation was determined between ESSPRI (TR) total score and salivary flow rate. In addition, the ESSPRI (TR) total score was correlated moderately and highly with all subdimensions and total scores of the OHIP-14 questionnaire. The ESSPRI (TR) total score was also determined to be correlated moderately and highly with all dimensions of OHRQOL-UK except symptom dimension. It can be considered that the relationship between the symptom dimension of OHRQOL-UK and the ESSPRI (TR) total score was not seen because the two questions evaluating the findings related to the symptoms in the symptom dimension of OHRQOL-UK did not show similarity to the ESSPRI (TR) questions.

The ESSPRI is an index that evaluates symptom severity of fatigue, pain and dryness. Fatigue, pain and dryness symptoms have a high impact on disease perception and quality of life. Disease activity evaluation results also provide information about the patient's systemic complications. In the reliability and validity study of the Brazil-Portuguese version of ESSPRI, the ESSPRI total score determining the severity of symptoms was found to be high in patients with low disease activity score. The authors emphasized that only evaluating disease activity does not provide enough information about the patient's symptoms and that ESSPRI should be evaluated.[19] However, a limitation of this study was that disease activity was not evaluated in the current study patients. If disease activity had been evaluated, systemic complications may have been revealed and the relationship between symptom severity and systemic complications could have been determined.

In conclusion, the ESSPRI (TR) is a valid and reliable index for assessing the severity of symptoms in patients with pSS.

Appendix 1. EULAR Sjögren Sendromu Hasta Bildirim İndeksi (ESSPRI)

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

The authors received no financial support for the research and/or authorship of this article.

References

1. Ramos-Casals M, Brito-Zerón P, Sisó-Almirall A, Bosch X. Primary Sjogren syndrome. BMJ 2012;344:e3821. DOI: 10.1136/bmj.e3821
2. Ramos-Casals M, Tzioufas AG, Font J. Primary Sjögren’s syndrome: new clinical and therapeutic concepts. Ann Rheum Dis 2005;64:347-54. DOI: 10.1136/ard.2004.025676
3. Porola P, Laine M, Virkki L, Poduval P, Konttinen YT. The influence of sex steroids on Sjögren's syndrome. Ann N Y Acad Sci 2007;1108:426-32. DOI: 10.1196/annals.1422.045
4. Mavragani CP, Moutsopoulos HM. The geoepidemiology of Sjögren’s syndrome. Autoimmun Rev 2010;9:A305-10. DOI: 10.1016/j.autrev.2009.11.004
5. Bloch KJ, Buchanan WW, Wohl MJ, Bunim JJ. Sjoegren’s syndrome. A clinical, pathological, and serological study of sixty-two cases. Medicine (Baltimore) 1965;44:187-231. DOI: 10.1097/00005792-196505000-00001
6. Kabasakal Y, Kitapcioglu G, Turk T, Oder G, Durusoy R, Mete N, et al. The prevalence of Sjögren's syndrome in adult women. Scand J Rheumatol 2006;35:379-83. DOI: 10.1080/03009740600759704
7. Kvarnström M, Ottosson V, Nordmark B, WahrenHerlenius M. Incident cases of primary Sjögren's syndrome during a 5-year period in Stockholm County: a descriptive study of the patients and their characteristics. Scand J Rheumatol 2015;44:135-42. DOI: 10.3109/03009742.2014.931457
8. Brito-Zerón P, Baldini C, Bootsma H, Bowman SJ, Jonsson R, Mariette X, et al. Sjögren syndrome. Nat Rev Dis Primers 2016;2:16047. DOI: 10.1038/nrdp.2016.47
9. Cornec D, Devauchelle-Pensec V, Mariette X, Jousse-Joulin S, Berthelot JM, Perdriger A, et al. Severe Health-Related Quality of Life Impairment in Active Primary Sjögren's Syndrome and Patient-Reported Outcomes: Data From a Large Therapeutic Trial. Arthritis Care Res (Hoboken) 2017;69:528-35. DOI: 10.1002/acr.22974
10. Ramos-Casals M, Font J. Primary Sjögren’s syndrome. In: Imboden J, Hellmann D, Stone J, editors. Current diagnosis and treatment in rheumatology. Upper Saddle River, NJ: McGraw-Hill; 2007. p. 237-45.
11. Stewart CM, Berg KM, Cha S, Reeves WH. Salivary dysfunction and quality of life in Sjögren syndrome: a critical oral-systemic connection. J Am Dent Assoc 2008;139:291-9. DOI: 10.14219/jada.archive.2008.0158
12. Zelles T, Purushotham KR, Macauley SP, Oxford GE, Humphreys-Beher MG. Saliva and growth factors: the fountain of youth resides in us all. J Dent Res 1995;74:1826-32. DOI: 10.1177/00220345950740120301
13. Bowman SJ, Pillemer S, Jonsson R, Asmussen K, Vitali C, Manthorpe R, et al. Revisiting Sjögren's syndrome in the new millennium: perspectives on assessment and outcome measures. Report of a workshop held on 23 March 2000 at Oxford, UK. Rheumatology (Oxford) 2001;40:1180-8. DOI: 10.1093/rheumatology/40.10.1180
14. Bowman SJ, Booth DA, Platts RG; UK Sjögren's Interest Group. Measurement of fatigue and discomfort in primary Sjogren's syndrome using a new questionnaire tool. Rheumatology (Oxford) 2004;43:758-64. DOI: 10.1093/rheumatology/keh170
15. Bowman SJ, Booth DA, Platts RG, Field A, Rostron J; UK Sjögren's Interest Group. Validation of the Sicca Symptoms Inventory for clinical studies of Sjögren's syndrome. J Rheumatol 2003;30:1259-66.
16. Vitali C, Palombi G, Baldini C, Benucci M, Bombardieri S, Covelli M, et al. Sjögren's Syndrome Disease Damage Index and disease activity index: scoring systems for the assessment of disease damage and disease activity in Sjögren's syndrome, derived from an analysis of a cohort of Italian patients. Arthritis Rheum 2007;56:2223-31. DOI: 10.1002/art.22658
17. Bowman SJ, Sutcliffe N, Isenberg DA, Goldblatt F, Adler M, Price E, et al. Sjögren's Systemic Clinical Activity Index (SCAI)--a systemic disease activity measure for use in clinical trials in primary Sjögren's syndrome. Rheumatology (Oxford) 2007;46:1845-51. DOI: 10.1093/rheumatology/kem280
18. Seror R, Ravaud P, Mariette X, Bootsma H, Theander E, Hansen A, et al. EULAR Sjogren's Syndrome Patient Reported Index (ESSPRI): development of a consensus patient index for primary Sjogren's syndrome. Ann Rheum Dis 2011;70:968-72. DOI: 10.1136/ard.2010.143743
19. Paganotti MA, Valim V, Serrano ÉV, Miyamoto ST, Giovelli RA, Santos MC. Validation and psychometric properties of the Eular Sjögren's Syndrome Patient Reported Index (ESSPRI) into Brazilian Portuguese. Rev Bras Reumatol 2015;55:439-45. DOI: 10.1016/j.rbre.2015.08.001
20. Hair JF, Black WC, Babin BJ, Anderson RE, editors. Multivariate Data analysis. 7th ed. New York: Pearson; 2014.
21. Patel R, Shahane A. The epidemiology of Sjögren's syndrome. Clin Epidemiol 2014;6:247-55. DOI: 10.2147/CLEP.S47399
22. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000;25:3186-91. DOI: 10.1097/00007632-200012150-00014
23. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005;8:94-104. DOI: 10.1111/j.1524-4733.2005.04054.x
24. Slade GD, Spencer AJ. Development and evaluation of the Oral Health Impact Profile. Community Dent Health 1994;11:3-11.
25. Mumcu G, Inanc N, Ergun T, Ikiz K, Gunes M, Islek U, et al. Oral health related quality of life is affected by disease activity in Behçet's disease. Oral Dis 2006;12:145-51. DOI: 10.1111/j.1601-0825.2005.01173.x
26. Naito M, Yuasa H, Nomura Y, Nakayama T, Hamajima N, Hanada N. Oral health status and health-related quality of life: a systematic review. J Oral Sci 2006;48:1-7. DOI: 10.2334/josnusd.48.1
27. Fidelix T, Czapkowski A, Azjen S, Andriolo A, Neto PH, Trevisani V. Low-level laser therapy for xerostomia in primary Sjögren's syndrome: a randomized trial. Clin Rheumatol 2018;37:729-36. DOI: 10.1007/s10067-017-3898-9
28. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 2016;15:155-63. DOI: 10.1016/j.jcm.2016.02.012