Reliability and validity of the Turkish version of Scleroderma Skin Patient-Reported Outcome in patients with systemic sclerosis

Abstract

Objectives: This study aimed to translate the Scleroderma Skin Patient-Reported Outcome (SSPRO) questionnaire to the Turkish (SSPRO-T) language and to assess its validity and reliability.

Patients and methods: Fifty-four systemic sclerosis (SSc) patients (51 females, 3 males; mean age: 49.8±10.4 years; range, 22 to 65 years) participated in the reliability and validity analysis between October 2022 and December 2022. The translation and cross-cultural adaptation of the SSPRO-T was applied in accordance with the procedure described by the Beaton guidelines. The SSPRO-T, the Scleroderma Health Assessment Questionnaire (SHAQ), the Health Assessment Questionnaire Disability Index (HAQ-DI), Skindex-29, and patient global skin severity were conducted in all participants for construct validity. The SSPRO-T was retested to assess its reliability after seven days.

Results: The SSPRO-T had a four-factor structure. The total SSPRO-T score and its subgroups correlated positively with SHAQ, HAQ-DI, Skindex-29, and patient global skin severity. The internal consistency and reliability were excellent in overall SSPRO-T and in the subgroups: physical effect, emotional effect, physical limitation, and social effect (Cronbach’s α=0.94, 0.80, 0.95, 0.93, and 0.84, respectively). The SSPRO-T had excellent test-retest reliability (r=0.91, p<0.001). In addition, no floor effect or ceiling effect was observed.

Conclusion: The SSPRO-T questionnaire is a reliable and valid tool and can be used in research and clinical practice in Turkish patients with SSc.

Introduction

Systemic sclerosis (SSc) is an autoimmune rheumatic disease characterized by fibrosis of the skin and internal organs and vasculopathy. Vasculopathy causes clinical symptoms related to structural vascular changes and links with inflammation and immune abnormalities in SSc.[1] Skin fibrosis, which may affect the skin throughout the body, is the other prominent feature of SSc and manifests almost in all patients with SSc at various severity. The skin involvement in SSc patients may result in a considerable decrease in quality of life (QoL) through emotional, physical, and social aspects.[2,3]

Skin involvement is assessed with various outcome measures in patients with SSc, and one of the most widely used outcome measure is the modified Rodnan skin score (mRSS).[4] The mRSS assesses the location and severity of skin fibrosis. However, functional and psychological alterations caused by skin fibrosis cannot be assessed via mRSS. Several previous studies have found that mRSS has no relationship with appearance self-esteem and is only weakly related to patients’ illness perception and psychosocial adjustment.[5,6] It is well known that skin involvement may lead to inadequate functionality in daily activities and mental or social problems. In addition, skin fibrosis may give rise to worries, anxiety, or depression.[7] Therefore, it is important to comprehensively evaluate the effects of skin fibrosis on QoL in SSc patients.

General health assessment and outcome measures related to organ involvement are widely used in SSc. The health questionnaires, such as the Health Assessment Questionnaire (HAQ)-Disability Index (DI),[8] the Short Form Health Survey-36 (SF-36),[3] and the Patient Reported Outcomes Measurement Information System-29 (PROMIS-29),[9] are validated in SSc patients; however, these questionnaires might not adequately evaluate disease-specific issues. Organ-specific outcome measures, such as the Mouth Handicap Scale,[10] the Symptom Burden Index,[11] and the scleroderma HAQ (SHAQ),[12] can be used to evaluate treatment response or to follow-up disease progression; however, they cannot reflect the effects of skin involvement on QoL.

A sole scale to objectively assess all physical, emotional, and social aspects of QoL in SSc patients is lacking. The Scleroderma Skin PatientReported Outcome (SSPRO) was developed by Man et al.[13] to investigate skin-related QoL in SSc patients. SSPRO has excellent reliability and validity in patients with SSc. Although SSPRO has been an important outcome measure to reflect the impact of skin involvement on QoL, it has not been investigated in Turkish. There is clearly a need for a tool to evaluate the effects of skin involvement on the QoL of the Turkish population. Hence, the purpose of the present study was to assess the cross-cultural adaptation, validity, and reliability of the Turkish version of the SSPRO (SSPRO-T).

Patients and Methods

The cross-sectional study was performed with 54 participants (51 females, 3 males; mean age: 49.8±10.4 years; range, 22 to 65 years) in the Rheumatology Clinic of the Ankara City Hospital between October 2022 and December 2022. Permission was obtained from the authors who developed the questionnaire. The translation and cross-cultural adaptation of the original version of the SSPRO into Turkish were performed in keeping with the procedure described by Beaton et al.[14] The SSPRO was independently translated from French to Turkish by three native Turkish speakers. Three Turkish native speakers, proficient in French and knowledgeable in the terminology of the questionnaire, compared the translated versions with the original text. Once all native speakers agreed that the translated versions were consistent, the three translations were blended into a single Turkish version. Two bilingual translators, both of whose first languages were French, worked independently to complete the back translation. With regard to the original French text, both back translators were in the dark. All translators formed the expert committee. To identify unclear or divergent questions, all translated versions were matched with the original French version. The expert committee came up with the SSPRO-T's final version, which was then approved by the committee’s members.

Inclusion criteria were determined as follows: (i) patients aged 18-65 years; (ii) being diagnosed with SSc for at least one month according to the American College of Rheumatology (ACR)/European Alliance of Associations for Rheumatology (EULAR) criteria;[15] (iii) being a Turkish native speaker and literate for providing informed consent and completing the outcome questionnaire; (iv) having knowledge of smartphone, tablet, or computer use for answering the questionnaires via online forms. Patients who had any additional dermatological disease except for scleroderma skin involvement, cognitive, or cooperation problems and patients who had an attack between a test and a retest of SSPRO-T were excluded.

Sociodemographic characteristics, such as age, height, body weight, marital status, education level, smoking status, and patient global skin severity, were evaluated in each patient. The disease duration, family history, and the mRSS of each patient were recorded by a rheumatologist. Modified RSS was used to assess skin thickness by palpation in patients with SSc as a primary outcome assessment in clinical trials and practice. Skin thickening was assessed by palpation at 17 skin sites and was graded on a semiquantitative scale with values of 0 (normal), 1 (mild), 2 (moderate), or 3 (severe skin thickening), scoring up to a maximum of 51 points. A higher score indicates greater extent and severity of skin thickening.[4]

An online link was composed on Google Forms for all questionnaires. The patients signed the informed consent form and marked the checkbox indicating that they were volunteering. Then, the patients submitted answers in the questionnaires, and the answers were automatically recorded in a spreadsheet. The following questionnaires were filled by all of the patients: SSPRO-T, HAQ-DI,[16] SHAQ-Visual Analog Scale (VAS),[12] and Skindex-29.[17] SSPRO-T was answered a second time within the next seven days.

The SSPRO was developed to evaluate skin-related QoL in SSc patients. The SSPRO consists of four domains (physical effect [PE], emotional effect [EE], physical limitation [PL], and social effect [SE]) and 18 items. The items are graded from no disability (0 points) to severe disability (6 points) on a 6-point scale. The scores are then summed to produce the final score (0 to 108 points). A lower score indicates a much better QoL. The questionnaire assesses the past four weeks. The SSPRO was found to have high reliability (Cronbach’s α=0.89-0.96) and construct validity.[13]

The HAQ-DI is a self-reported scale with 20 items in eight domains that measures the difficulty of executing activities of daily living: getting dressed, getting up, eating, walking, maintaining hygiene, gripping, reaching, and routine daily activities.[8] Each item is scored on a 0-3 scale, with 0 denoting “without difficulty” and 3 denoting “unable to perform,” and extra points can be added if aids or equipment are required for particular activities. A higher score shows worse functionality.[8,18] The Turkish version of the HAQ-DI has excellent reliability (Cronbach’s α=0.97) and external construct validity.[16]

The SHAQ includes five questions related to overall disease severity. The questions are evaluated on a VAS with a length of 15 cm. “Does not interfere” and “very severe limitations” are the two endpoints of the line. The value is multiplied by 0.2 to produce the final VAS score. The score is between 0 and 3, with 0 being the minimum and 3 being the maximum limitation.[12]

Skindex-29 is a self-reported questionnaire to assess dermatology-specific QoL including 29 items in three domains (symptom, emotion, and function). Each item is graded on a scale of 0 to 100 (0 points=never, 25 points=rarely, 50 points=occasionally, 75 points=usually, and 100 points=always). A higher score demonstrates worse QoL.[17,19]

Statistical analysis

The sample size of the study was calculated with the G*Power version 3.1.9.2 (HeinrichHeine-Universität Düsseldorf, Düsseldorf, Germany). The SSPRO total score was chosen with an intraclass correlation coefficient (ICC) in agreement with the study by Man et al.[13] (ICC=0.82, effect size=0.31, and medium effect size). A sample size of at least 54 was identified, providing a power of 80% and a=0.05 (two-tailed).

The statistical analyses were performed with the IBM SPSS version 22.00 software (IBM Corp., Armonk, NY, USA). Descriptive statistics were expressed as mean ± standard deviation, and frequency (%). The Kolmogorov– Smirnov test was applied to find whether the numeric variables conform to the normal distribution. Principal components analysis was performed for construct validity, and the Kaiser-Meyer-Olkin (KMO) sample test and Bartlett’s test were used to check the suitability of the data for factor analysis. Internal consistency was checked using Cronbach’s alpha and the ICC. Cronbach's alpha was regarded as very good if the coefficient was >0.80. The ICC values >0.80 indicate excellent reliability. The maximum and minimum scores of the scale were found and their percentages were calculated to determine possible floor effects and ceiling effects. The Pearson correlation analysis was performed for the item-total score analysis and for the relationship between test-retest reliability and test-retest scores. The Pearson correlation analysis was interpreted as excellent, very good, good, weak, and inadequate (0.81-1.00, 0.61-0.80, 0.41-0.60, 0.21-0.40, and 0-0.20, respectively). A paired samples t-test was used to compare the testretest mean score. The critical significance level was determined as p<0.05.

Results

To assess the test-retest reliability, 36 participants whose clinical status did not change were asked to fill out the SSPRO-T again the following seven days after the first assessment (Figure 1). The mean age of diagnosis was 37.2±15.2 years, and the mean Raynaud's phenomenon age of onset was 38.9±12.6 years (Table 1).

Table 1. Patients characteristics at baseline

Figure 1. Flow diagram of the patients.

SSPRO-T demonstrated adequacy of sample for explanatory factor analysis (EFA) with a KMO coefficient of 0.869 and Bartlett's sphericity value with statistical significance (chi-square=912.462, degree of freedom=153, p<0.001). According to the EFA result, the SSPRO-T consistently demonstrated four distinct dimensions (Figure 2). The factor's explanation variance was 78.48%.

Figure 2. Principal component analysis, scree plot.

Cronbach’s alpha value of the SSPRO-T was 0.94. This result demonstrated that the questionnaire had a very good and adequate internal consistency. Test-retest ICC value of the total score and each subgroup varied between 0.91 (95% confidence interval [CI]: 0.82-0.95, p<0.001) and 0.68 (95% CI: 0.46-0.82, p<0.001). Test-retest correlation of the total score was 0.91 (Table 2). Excellent levels of reliability for both the total score and each subgroup score of the SSPRO-T were demonstrated (ICC>0.80). However, there was good reliability in the SE subgroup. A floor effect was found only in the SE subgroup. Moreover, total score, PE subgroup, EE subgroup, and PL subgroup did not have floor (0%) and ceiling (0%) effects (Table 2).

Table 2. SSPRO-T descriptive statistics, internal consistency, and test-retest reliability

The test-retest reliability of the SSPRO-T was significant and excellent at 0.91 (Table 3). The associations between the SSPRO-T and other questionnaires (mRSS, HAQ-DI, SHAQ-VAS, Skindex-29, and patient global skin severity) were examined for construct validity. The Pearson correlation coefficients between the total scores of the SSPRO-T and the HAQ-DI (r=0.58, p<0.001), SHAQ-VAS (r=0.74, p<0.001), Skindex-29 (r=0.83, p<0.001), and patient global skin severity (r=0.59, p<0.001) were statistically significant. However, the Pearson correlation coefficient between the total score of the SSPRO-T scale and the total score of physicians assessed skin scores (mRSS) was not statistically significant (r=0.03, p>0.05).

Table 3. Association of SSPRO with its subgroups and other outcome measures

Discussion

The mRSS is the most commonly used scale to assess skin involvement in patients with SSc.[4] However, this scale only assesses the extent and severity of skin fibrosis. It does not evaluate the physical and mental effects of skin involvement in patients. There are many patient-reported outcome questionnaires that have reliable and valid Turkish versions, including PROMIS-29, HAQ-DI, and SF-36. These scales are applied in the evaluation of patients in research and clinical practice in SSc. However, none of them are specific to SSc.[3,9,16]

Hence, this study investigated the validity and reliability of SSPRO-T in patients with SSc and determined that the SSPRO-T was a valid, reliable, suitable, and multidimensional scale for the Turkish population. Cronbach’s alpha value of SSPRO-T was 0.94, demonstrating an excellent internal consistency. The result is similar to the original questionnaire, which indicated a Cronbach’s alpha of 0.96. Moreover, Cronbach’s alpha values for the subgroup dimensions in the original version were as follows: PE, 0.89; EE, 0.95; PL, 0.92; SE, 0.92.13 The Cronbach’s alpha for dimensions in our study was 0.80 for the PE, 0.95 for the EE, 0.93 for the PL, and 0.84 for SE. These results indicate that SSPRO-T has a similarity with the original version and excellent internal consistency.

The test-retest median interval of the original SSPRO was two weeks. However, Marx et al.[20] stated that there was no significant difference in test-retest reliability when analyzed in a range of two days to two weeks. In the light of this result, the test-retest analysis of the SSPRO-T was applied with a time interval of one week. In the original version, Man et al.[13] determined that the test-retest ICC value of the total score was 0.82. Similarly, the test-retest ICC value of the total score was 0.91 (0.82–0.95) in our study. The test-retest ICC value of subgroup dimensions were 0.88 (0.77-0.93) for PE, 0.90 (0.81-0.94) for EE, 0.81 (0.62-0.90) for PL, and 0.68 (0.46-0.82) for SE of the scale. These results demonstrate that SSPRO-T is a reliable questionnaire.

The KMO test and Bartlett’s test are used to show whether the study sample is suitable and sufficient for factor analysis. In our study, they indicated that the sample size was suitable and adequate for factor analysis. The scree plot and EFA demonstrated that SSPRO-T is a suitable four-factor scale to evaluate PE (Items 1-5), EE (Items 6-11), PL (Items 12-15), and SE (Items 16-18). The factor analysis was similar to the original version.[13] The floor effect or ceiling effect confirming the content validity of the Turkish version supports the validity of SSPRO-T for the evaluation of the skin in SSc. The present study indicated that there were no floor or ceiling effects in the SSPRO-T questionnaire.

Construct validity was verified with significant correlation between SSPRO-T and its subgroups with all outcome measures, except for mRSS. An excellent positive correlation between SSPRO-T and Skindex-29, a good positive correlation between SSPRO-T and HAQ-DI as well as patient global skin severity, and a very good positive correlation between SSPRO-T and SHAQ-VAS were reported. The construct validity was similar to the original version.[13] However, only the mRSS scale did not correlate with SSPRO-T and the subgroups (r=0.03, p>0.05). The mRSS scale only assesses the extent and severity of fibrosis. It does not evaluate the physical and mental effects of skin fibrosis, whereas the SSPRO-T questionnaire assesses PE, EE, and SE. This result emphasizes that patient perception regarding disease severity and QoL may not be proportionate with true disease severity or clinician’s evaluation, which is a common phenomenon in chronic rheumatic disorders.

There are several limitations to our study. First, responsiveness, which is the sensitivity of a scale to clinical changes, was not assessed in our study. Responsiveness of the SSPRO-T should be examined in future studies. The SSPRO scale was developed to measure skin-related disability. However, patients with other manifestations, such as Raynaud's phenomenon, arthritis, and major organ involvement, were also included in the present study. The confounding effects of other involvements were not investigated in our study.

In conclusion, this study demonstrated that the SSPRO-T is a reliable and valid tool to assess the physical, emotional, social, and physical deteriorations in Turkish SSc patients due to skin fibrosis. The temporal changes in SSPRO-T, the performance in different disease forms (diffuse and limited groups) of SSc and effects of other disease manifestations may be aspects to be investigated in future studies.

Citation: Sari F, Bazancir Apaydin Z, Apaydin H, Kayaalp M, Erden A, Güven SC, et al. Reliability and validity of the Turkish version of Scleroderma Skin Patient-Reported Outcome in patients with systemic sclerosis. Arch Rheumatol 2024;39(1):52- 59. doi: 10.46497/ArchRheumatol.2023.10183.

The study protocol was approved by the Ankara City Hospital Clinical Research Ethics Committee Ethics Committee (date: 12.10.2022, no: E2-22-2567). The study was conducted in accordance with the principles of the Declaration of Helsinki.

A written informed consent was obtained from each patient.

Concept: F.S., Z.B.A.; Design: F.S., Z.B.A., H.A., S.C.G., B.A., A.E., O.K.; Supervision: F.S., Z.B.A., H.A., O.K.; Data collection and/or processing: F.S., Z.B.A., H.A., M.K., S.C.G., B.A., A.E., A.O.; Analysis and/or interpretation: F.S., Z.B.A.; Literature review: F.S., Z.B.A., H.A.; Writing: F.S., Z.B.A.; Critical review: F.S., Z.B.A., H.A., M.K., S.C.G., B.A., A.E., A.O., O.K., .E.

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

The authors received no financial support for the research and/or authorship of this article.

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

1. Asano Y, Sato S. Vasculopathy in scleroderma. Semin Immunopathol 2015;37:489-500. doi: 10.1007/ s00281-015-0505-5. DOI: 10.1007/s00281-015-0505-5
2. Hudson M, Thombs BD, Steele R, Panopalis P, Newton E, Baron M; Canadian Scleroderma Research Group. Health-related quality of life in systemic sclerosis: A systematic review. Arthritis Rheum 2009;61:1112-20. doi: 10.1002/art.24676. DOI: 10.1002/art.24676
3. Li L, Cui Y, Chen S, Zhao Q, Fu T, Ji J, et al. The impact of systemic sclerosis on health-related quality of life assessed by SF-36: A systematic review and meta-analysis. Int J Rheum Dis 2018;21:1884-93. doi: 10.1111/1756-185X.13438. DOI: 10.1111/1756-185X.13438
4. Clements P, Lachenbruch P, Siebold J, White B, Weiner S, Martin R, et al. Inter and intraobserver variability of total skin thickness score (modified Rodnan TSS) in systemic sclerosis. J Rheumatol 1995;22:1281-5.
5. Malcarne VL, Hansdottir I, McKinney A, Upchurch R, Greenbergs HL, Henstorf GH, et al. Medical signs and symptoms associated with disability, pain, and psychosocial adjustment in systemic sclerosis. J Rheumatol 2007;34:359-67.
6. van Lankveld WG, Vonk MC, Teunissen H, van den Hoogen FH. Appearance self-esteem in systemic sclerosis--subjective experience of skin deformity and its relationship with physician-assessed skin involvement, disease status and psychological variables. Rheumatology (Oxford) 2007;46:872-6. doi: 10.1093/rheumatology/kem008. DOI: 10.1093/rheumatology/kem008
7. Baubet T, Brunet M, Garcia De La Peña-Lefebvre P, Taïeb O, Moro MR, Guillevin L, et al. Psychiatric manifestations of systemic sclerosis. Ann Med Interne (Paris) 2002;153:237-41.
8. Johnson SR, Hawker GA, Davis AM. The health assessment questionnaire disability index and scleroderma health assessment questionnaire in scleroderma trials: An evaluation of their measurement properties. Arthritis Rheum 2005;53:256-62. doi: 10.1002/art.21084. DOI: 10.1002/art.21084
9. Hinchcliff ME, Beaumont JL, Carns MA, Podlusky S, Thavarajah K, Varga J, et al. Longitudinal evaluation of PROMIS-29 and FACIT-dyspnea short forms in systemic sclerosis. J Rheumatol 2015;42:64-72. doi: 10.3899/jrheum.140143. DOI: 10.3899/jrheum.140143
10. Mouthon L, Rannou F, Bérezné A, Pagnoux C, Arène JP, Foïs E, et al. Development and validation of a scale for mouth handicap in systemic sclerosis: The Mouth Handicap in Systemic Sclerosis scale. Ann Rheum Dis 2007;66:1651-5. doi: 10.1136/ ard.2007.070532. DOI: 10.1136/ard.2007.070532
11. Kallen MA, Mayes MD, Kriseman YL, de Achaval SB, Cox VL, Suarez-Almazor ME. The symptom burden index: Development and initial findings from use with patients with systemic sclerosis. J Rheumatol 2010;37:1692-8. doi: 10.3899/jrheum.090504. DOI: 10.3899/jrheum.090504
12. Temiz Karadag D, Karakas F, Tekeoglu S, Yazici A, Isik OO, Cefle A. Validation of Turkish version of the scleroderma Health Assessment Questionnaire. Clin Rheumatol 2019;38:1917-23. doi: 10.1007/ s10067-019-04494-5. DOI: 10.1007/s10067-019-04494-5
13. Man A, Correa JK, Ziemek J, Simms RW, Felson DT, Lafyatis R. Development and validation of a patient-reported outcome instrument for skin involvement in patients with systemic sclerosis. Ann Rheum Dis 2017;76:1374-80. doi: 10.1136/ annrheumdis-2016-210534. DOI: 10.1136/annrheumdis-2016-210534
14. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000;25:3186-91. doi: 10.1097/00007632- 200012150-00014. DOI: 10.1097/00007632-200012150-00014
15. van den Hoogen F, Khanna D, Fransen J, Johnson SR, Baron M, Tyndall A, et al. 2013 classification criteria for systemic sclerosis: An American college of rheumatology/European league against rheumatism collaborative initiative. Ann Rheum Dis 2013;72:1747- 55. doi: 10.1136/annrheumdis-2013-204424. DOI: 10.1136/annrheumdis-2013-eular.238
16. Küçükdeveci AA, Sahin H, Ataman S, Griffiths B, Tennant A. Issues in cross-cultural validity: Example from the adaptation, reliability, and validity testing of a Turkish version of the Stanford Health Assessment Questionnaire. Arthritis Rheum 2004;51:14-9. doi: 10.1002/art.20091. DOI: 10.1002/art.20091
17. Aksu AE, Urer MS, Sabuncu I, Saracoglu ZN, Chren MM. Turkish version of Skindex-29. Int J Dermatol 2007;46:350-5. doi: 10.1111/j.1365- 4632.2007.03008.x. DOI: 10.1111/j.1365-4632.2007.03008.x
18. Pope J. Measures of systemic sclerosis (scleroderma): Health Assessment Questionnaire (HAQ) and Scleroderma HAQ (SHAQ), physician- and patientrated global assessments, Symptom Burden Index (SBI), University of California, Los Angeles, Scleroderma Clinical Trials Consortium Gastrointestinal Scale (UCLA SCTC GIT) 2.0, Baseline Dyspnea Index (BDI) and Transition Dyspnea Index (TDI) (Mahler's Index), Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR), and Raynaud's Condition Score (RCS). Arthritis Care Res (Hoboken) 2011;63 Suppl 11:S98-111. doi: 10.1002/acr.20598. DOI: 10.1002/acr.20598
19. Chren MM, Lasek RJ, Quinn LM, Mostow EN, Zyzanski SJ. Skindex, a quality-of-life measure for patients with skin disease: Reliability, validity, and responsiveness. J Invest Dermatol 1996;107:707-13. doi: 10.1111/1523-1747.ep12365600. DOI: 10.1111/1523-1747.ep12365600
20. Marx RG, Menezes A, Horovitz L, Jones EC, Warren RF. A comparison of two time intervals for testretest reliability of health status instruments. J Clin Epidemiol 2003;56:730-5. doi: 10.1016/s0895- 4356(03)00084-2. DOI: 10.1016/S0895-4356(03)00084-2