Advertisement

Electronic Version of the EQ-5D Quality-of-Life Questionnaire: Adaptation to a Brazilian Population Sample

Open ArchivePublished:May 11, 2018DOI:https://doi.org/10.1016/j.vhri.2017.11.002

      Abstract

      Objectives

      To assess the measurement equivalence of the original paper version of an adapted tablet version of the EuroQol five-dimensional questionnaire (EQ-5D).

      Methods

      A randomly selected sample of 509 individuals aged 18 to 64 years from the general population responded to the EQ-5D at two time points separated by a minimum interval of 24 hours and were allocated to one of the following groups: test-retest group (tablet-tablet) or crossover group (paper-tablet and tablet-paper). Agreement between methods was determined using the intraclass correlation coefficient (ICC) and the κ coefficient.

      Results

      In the crossover group, the following ICC values were obtained: 0.76 (confidence interval [CI] 0.58–0.89) for EQ-5D scores and 0.77 (CI 0.68–0.84) for visual analogue scale in subjects responding first to the tablet version; 0.83 (CI 0.75–0.89) for EQ-5D scores and 0.75 (CI 0.67–0.85) for visual analogue scale in subjects responding first to the paper version. In the test-retest group, the ICC was 0.85 (CI 0.73–0.91) for EQ-5D scores and 0.79 (CI 0.66–0.87) for visual analogue scale. The κ values were higher than 0.69 in this group. The internal consistencies of the paper and tablet methods were similar.

      Conclusions

      The paper and tablet versions of the EQ-5D are equivalent. Test-retest and crossover agreement was high and the acceptability of the methods was similar among individuals.

      Keywords

      Introduction

      A patient’s point of view regarding his own health has been valued to quantitate the impact of a given condition or treatment on the life of a person. Thus, quality of life has been considered to be an important and acceptable outcome offering a new perspective in addition to the traditional morbidity and mortality outcomes [
      • Campolina A.G.
      • Ciconelli R.M.
      Qualidade de vida e medidas de utilidade: parâmetros clínicos para as tomadas de decisão [Spanish].
      ,
      • Lenert L.
      • Kaplan R.M.
      Validity and interpretation of preference-based measures of health-related quality of life.
      ]. Quality of life is used in economic analyses of cost-utility and is represented by a measure called quality-adjusted life-year (QALY) [
      • Drummond M.F.
      ], which adds utility scores to the years of life lived in a determined health status.
      Utility scores are a type of measure of quality of life represented by a single value that expresses the preference of persons for certain health states and are appropriate for use in economic analyses. The EuroQol five-dimensional questionnaire (EQ-5D), developed by the EuroQol Group [
      • Szende A.
      • Oppe M.
      • Devlin N.J.
      EQ-5D Value Sets: Inventory, Comparative Review and User Guide.
      ], is an example of a quality-of-life questionnaire that generates utility scores and provides a simple and generic measure of health. The translation and cultural adaptation of the Brazilian Portuguese version of the questionnaire was carried out according to the norms established by the translation group of EuroQol in 2002 [
      EuroQol Group
      EuroQol—a new facility for the measurement of health-related quality of life.
      ,
      • Brooks R.
      • Rabin R.
      • de Charro F.
      The Measurement and Valuation of Health Status Using EQ-5D: A European Perspective.
      ]. The valuation of the health states generated by the questionnaire has also been performed for the Brazilian population between 2012 and 2013 [
      • Viegas Andrade M.
      • Noronha K.
      • Kind P.
      • et al.
      Societal preferences for EQ-5D health states from a Brazilian population survey. Value Health Reg.
      ,
      • Santos M.
      • Cintra M.A.
      • Monteiro A.L.
      • et al.
      Brazilian valuation of EQ-5D-3L health states: results from a saturation study.
      ].
      The National Institute for Health and Care Excellence of the United Kingdom recommends that all economic assessments in health should include evaluation using QALY as a measure of health benefits, preferentially using the EQ-5D for collection [
      • Tosh J.C.
      • Longworth L.J.
      • George E.
      Utility values in National Institute for Health and Clinical Excellence (NICE) technology appraisals.
      ]. The cost-effectiveness panel likewise recommends that economic health assessments should include measurements of benefits that take into consideration the QALY [
      • Weinstein M.C.
      • Siegel J.E.
      • Gold M.R.
      • et al.
      Recommendations of the panel on cost-effectiveness in health and medicine.
      ,
      • Brauer C.A.
      • Rosen A.B.
      • Greenberg D.
      • Neumann P.J.
      Trends in the measurement of health utilities in published cost-utility analyses.
      ].
      An alternative to the traditional application of quality-of-life questionnaires on paper is the use of electronic media such as tablets, which are being increasingly used and tested to replace the paper versions. Electronic media have advantages over paper versions because greater reliability of the generated data is expected from the automatic creation of a database, easy storage, and a greater control of the date and time of response to the questionnaire. In addition, digital media can prevent some frequent errors occurring in handwritten questionnaires such as duplicate marking, lack of response marking, and illegible responses [
      • Bushnell D.M.
      • Martin M.L.
      • Scanlon M.
      • et al.
      Equivalence and measurement properties of an electronic version of the Psoriasis Symptom Inventory.
      ,
      • Gwaltney C.J.
      • Shields A.L.
      • Shiffman S.
      Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review.
      ]. Technology based on portable computers or electronic devices can reduce the impact of data collection for both the interviewer and the respondents, facilitating rapid access to the collected information and generating lower application costs [
      • Norman R.
      • King M.T.
      • Clarke D.
      • et al.
      Does mode of administration matter? Comparison of online and face-to-face administration of a time trade-off task.
      ].
      To use electronic versions of questionnaires, it is necessary to demonstrate that the instrument can generate responses equivalent to those obtained with the standard paper format. The International Society for Pharmacoeconomics and Outcomes Research Group [
      • Coons S.J.
      • Gwaltney C.J.
      • Hays R.D.
      • et al.
      Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force Report.
      ] states that for instruments of subjective outcome measurement, electronic media collection must provide equivalent data with respect to the reliability of the generated data and also measurement equivalence compared with traditional paper methods. The group also points out that the changes made in the format of a questionnaire so as to apply it with media other than the standard one should be minimal [
      • Coons S.J.
      • Gwaltney C.J.
      • Hays R.D.
      • et al.
      Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force Report.
      ].
      Several studies of generic and specific instruments that measure quality of life have reported equivalence of the results obtained with paper-based and touch-screen methods. These studies have demonstrated psychometric comparability in terms of reliability and validity, as well as good acceptability of the electronic versions; for example, Bushnell et al. evaluated the equivalence of the electronic and paper versions of the EQ-5D, Chang et al. carried out similar analyses regarding a specific EORTC quality-of-life questionnaire for patients with prostate cancer (EORTC QLQ-PR25), and Asheley et al. did the same with the Social Difficulty Inventory [
      • Bushnell D.M.
      • Martin M.L.
      • Scanlon M.
      • et al.
      Equivalence and measurement properties of an electronic version of the Psoriasis Symptom Inventory.
      ,
      • Bushnell D.M.
      • Martin M.L.
      • Parasuraman B.
      Electronic versus paper questionnaires: a further comparison in persons with asthma.
      ,
      • Bushnell D.M.
      • Reilly M.C.
      • Galani C.
      • et al.
      Validation of electronic data capture of the Irritable Bowel Syndrome–Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol.
      ,
      • Chang Y.J.
      • Chang C.H.
      • Peng C.L.
      • et al.
      Measurement equivalence and feasibility of the EORTC QLQ-PR25: paper-and-pencil versus touch-screen administration.
      ,
      • Ashley L.
      • Keding A.
      • Brown J.
      • et al.
      Score equivalence of electronic and paper versions of the Social Difficulties Inventory (SDI-21): a randomised crossover trial in cancer patients.
      ,
      • Bushnell D.M.
      • Martin M.L.
      • Ricci J.F.
      • et al.
      Performance of the EQ-5D in patients with irritable bowel syndrome.
      ].
      The tablet version of the EQ-5D has been translated into 100 languages by the EuroQol Group, including the Brazilian Portuguese language. Nevertheless, so far, no studies have assessed the equivalence of the paper and tablet versions in Brazil. A US study tested the equivalence of the tablet and paper versions of the EQ-5D in a sample of patients with irritable bowel syndrome, and another tested only the visual analogue scale (VAS) of the instrument in a population sample from Arizona [
      • Bushnell D.M.
      • Martin M.L.
      • Ricci J.F.
      • et al.
      Performance of the EQ-5D in patients with irritable bowel syndrome.
      ,
      • Ramachandran S.
      • Lundy J.J.
      • Coons S.J.
      Testing the measurement equivalence of paper and touch-screen versions of the EQ-5D visual analog scale (EQ VAS).
      ].
      The objective of the present study was to assess the equivalence of the tablet and paper versions of the EQ-5D, both in Brazilian Portuguese, in the general population from two distinct regions of Brazil.

      Methods

       Study Design

      A population-based cross-sectional study was conducted on literate subjects aged 18 to 64 years. The characteristics of the sample in terms of age and educational level were the same as those applied in the sampling of the study of valuation of health status of the EQ-5D for the Brazilian population, because the health status values obtained in the aforementioned study were the same as those used to calculate the EQ-5D scores in the present study. The Brazilian study of valuation considered only literate subjects because its research protocol contained complex techniques such as time trade-off, whose application to illiterate subjects is quite limited. In addition, the EuroQol Group uses specific versions of the questionnaire for minors and for the elderly and the version used in the present study was that for adults up to 64 years. The sample was stratified into two age groups (one from 18 to 49 years and the other from 50 to 64 years) and into four educational levels. The subjects were selected at random in three regions with different economic levels, according to Brazilian Institute of Geography and Statistics (IBGE) data, from the cities of Porto Alegre and Belo Horizonte, for a total of 509 subjects.
      On the basis of the recommendations for equivalence studies, the subjects were divided into two randomized groups, one of them for test-retest assessment of the instrument using only electronic media (tablet-tablet sample) and the other as the crossover group, half of which responded to the tablet version and then to the traditional paper system (tablet-paper sample) and the other half to the paper system and then to the tablet [
      • Coons S.J.
      • Gwaltney C.J.
      • Hays R.D.
      • et al.
      Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force Report.
      ] (paper-tablet sample). A washout period of at least 24 hours and at most of 7 days was planned between applications in an attempt to eliminate a possible carryover effect of learning on the instrument applied, that is, to avoid the possibility that the responses of the subject in the second application might be influenced by the memory of the responses given in the first application, possibly demonstrating a capacity of the subject’s memory more than the capacity of the instrument to remain stable over repeated applications. The test-retest group was used to assess only the tablet method’s reliability because the paper version of the EQ-5D has already been tested and validated in other studies, including the Brazilian Portuguese version. Figure 1 is a schematic illustration of the sampling methodology used.
      Fig. 1
      Fig. 1Randomized design, data capture. EQ-5D, EuroQol five-dimensional questionnaire.

       Research Protocol

      The EQ-5D is a self-applicable instrument that consists of two parts. The first is descriptive, presenting five health dimensions: mobility, personal care, habitual activities, pain/malaise, and anxiety/depression, each having three levels of severity, generating 243 possible health states. The second contains a VAS ranging from 0 (worst imaginable health status) to 100 (best imaginable health status). All participants responded to the EQ-5D and to a sociodemographic questionnaire. In the application of the electronic media, the subjects initially responded to “test questions” to test their touch-screen ability so that they would understand how to mark the VAS and the remaining questions.
      The sociodemographic questionnaire contained the following items: presence of chronic diseases assessed by the question “Has any health professional (doctor or nurse) stated that you have or had one of these diseases?”; schooling (of the research subject himself and the highest educational level in his residence); frequency of the use of electronic media; sex; age; marital status; and social class based on Brazilian economic classification criteria that take into account all items running in the house, education of owners, and access to utilities. In the second application, the subject was asked about his facility in responding to the tablet and paper questionnaires with the following question: “Do you consider that responding to this questionnaire on a tablet was …” (when he responded on a tablet) or “Do you consider that responding to this questionnaire on paper was …” (when he responded on paper), with responses in a categorical format of “very easy,” “easy,” “difficult,” and “very difficult.” The subject was also asked to compare the ease of application of the two methods. In addition, the subject was questioned about whether any negative fact had occurred in his life between the applications of the questionnaire and whether his health status at that time compared with his health status on the occasion of the first application was equal, better, much better, worse, or much worse. The social class of the subjects was classified using the criteria of the economic classification of the Brazilian Association of Research Companies (ABEP), which consider schooling, sanitation, and consumer goods in the residence [

      Pesquisa A-ABdEd. Critério de classificação econômica. Brazil, 2014.

      ].

       Statistical Analysis

      Data were analyzed statistically using the SPSS software version 19 (IBM Corp., Armonk, NY) and the R program (R Development Core Team, Vienna, Austria). The design used involved a test-retest group and a crossover group. The test-retest design was used to assess the temporal stability of the electronic media, and the crossover design was used for the participants to be their own controls so as to determine the agreement between methods [
      • Coons S.J.
      • Gwaltney C.J.
      • Hays R.D.
      • et al.
      Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force Report.
      ,
      • Urbina S.
      Fundamentos da Testagem Psicológica.
      ].
      The intraclass correlation coefficient (ICC) for continuous measures was used to determine the agreement between methods and applications. The ICC estimates the total variability of the measures due to interindividual variations, considering a minimum acceptable value of 0.70, which indicates moderate agreement [
      • Switzer G.E.
      • Wisniewski S.R.
      • Belle S.H.
      • et al.
      Selecting, developing, and evaluating research instruments.
      ]. For dichotomous or ordinal measures, we used the κ coefficient, which is based on the number of concordant responses and measures the degree of agreement beyond what would be expected at random; values higher than 0.60 are considered good [
      • Urbina S.
      Fundamentos da Testagem Psicológica.
      ]. The Cronbach α was used to assess reliability by means of the internal consistency of the methods. The minimum acceptable α value is 0.70 and values ranging from 0.80 to 0.90 are considered good. The Cronbach α is affected by the number of items that are part of a scale [
      • Aaronson N.1
      • Alonso J.
      • Burnam A.
      • et al.
      Assessing health status and quality-of-life instruments: attributes and review criteria.
      ,
      • Cronbach L.
      Coefficient alpha and the internal structure of tests.
      ]. Mixed models were also used to assess the carryover effect. In addition, the t test and χ2 test were used to compare the sociodemographic data of the subjects with and without changes in health status. The level of significance was set at 0.05 in all analyses.

      Results

       Quality of the Data

      The paper questionnaires were entered into the computer with a double entry and with the verification of discrepancies. About 92 (15.80%) interviews were discarded because of data inconsistencies and refusal to submit the second application and so these were replaced by new collections to reach the planned sample number of 518 subjects.
      The file containing the paper applications was joined with the file of the tablet applications. In this phase, the inconsistencies were determined once more by the investigators, who observed the date and time of collection, with the questionnaires that had been re-applied within less than 24 hours or more than 7 days being excluded from analysis (1.72%). The final database contained 509 (98.26%) subjects.

       Characteristics of the Sample

      Of the 509 subjects, only those who responded that their health status continued to be the same as that reported in the first application were included in the analyses, for a total of 354 subjects. This decision was based on the fact that changes in EQ-5D scores would reflect the real alteration in health status and not the instability of the instrument. This type of analysis has already been carried out in other studies of the equivalence of the electronic version of the EQ-5D [
      • Bushnell D.M.
      • Reilly M.C.
      • Galani C.
      • et al.
      Validation of electronic data capture of the Irritable Bowel Syndrome–Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol.
      ].
      A total of 155 subjects reported changes in health status. Thus, an analysis was carried out to compare the sociodemographic differences between the groups with and without changes in health status. The results showed that significantly more women reported changes in health status, with a greater representation of sociodemographic classes C1 and C2 and a lower representation of classes A, B1, and B2, and a higher frequency of depression, arthritis, and cirrhosis. No significant differences were detected regarding other diseases, age, or schooling.
      Table 1 lists the sociodemographic characteristics of the three samples. For the analysis of the sociodemographic data, we pre-established as standard the responses to the paper questionnaire for the crossover group and the responses to the first application of the tablet for the test-retest group.
      Table 1Sociodemographic characteristics of the subjects according to the method applied
      CharacteristicGroup
      CrossoverTest-retest
      Paper-tablet (N = 118)Tablet-paper (N = 127)Tablet-tablet (N = 109)
      Sex, female, n (%)81 (68.6)71 (55.9)63 (57.8)
      Age (y), mean ± SD42.81 ± 14.4141.74 ± 14.044.38 ± 13.59
      Schooling, n (%)
       Incomplete elementary13 (11.0)17 (13.4)14 (12.8)
       Complete elementary25 (21.2)21 (16.5)25 (22.9)
       Complete middle school30 (25.4)42 (33.1)32 (29.4)
       Complete high school27 (22.9)28 (22.0)21 (19.3)
       Incomplete higher education9 (7.6)8 (6.3)8 (7.3)
       Complete higher education14 (11.9)11 (8.7)9 (8.2)
      Chronic diseases, n (%)
       Hypertension45 (38.1)40 (31.5)34 (31.2)
       Diabetes14 (11.9)15 (11.8)8 (7.3)
       Depression/anxiety28 (23.7)26 (20.5)28 (25.7)
       Respiratory problems20 (16.9)30 (23.6)29 (26.6)
       Spinal disorders21 (17.8)16 (12.6)21 (19.3)
       Arthritis12 (10.2)10 (7.9)11 (10.1)
       Heart disease13 (11.0)10 (7.9)12 (11.0)
       Kidney failure9 (7.6)6 (4.7)6 (5.5)
       Cancer4 (3.4)5 (3.9)2 (1.8)
       Tuberculosis2 (1.7)1 (0.8)3 (2.8)
       Cirrhosis0 (0)0 (0)1 (0.9)
       HIV/AIDS1 (0.8)0 (0)3 (2.8)
      Social class, n (%)
       A5 (4.2)7 (5.5)6 (5.5)
       B19 (7.6)11 (8.7)7 (6.4)
       B231 (26.3)25 (19.7)27 (24.8)
       C137 (31.4)39 (30.7)36 (33.0)
       C231 (26.3)34 (26.8)26 (23.9)
       D–E5 (4.2)11 (8.7)7 (6.4)
      Females predominated in the three samples, with a greater predominance in the paper-tablet crossover sample. Mean age ranged from 41 to 44 years. The predominant educational level in all groups was “complete middle and high school,” representing 48.3% of the paper-tablet samples, 55.1% of the tablet-paper samples, and 48.7% of the tablet-tablet samples. The most predominant chronic disease in all samples was systemic arterial hypertension, affecting 31% to 38% of the subjects, followed by depression and anxiety, affecting 20% to 26% of the study population.

       Equivalence Analysis in the Crossover Group

      In the crossover design, we assessed agreement using the ICC and detected a satisfactory correlation in the VAS and the paper-tablet sample and also in the tablet-paper sample. The paper-tablet sample showed a very good correlation for the utility scores and the tablet-paper sample showed a satisfactory correlation (Table 2).
      Table 2EQ-5D crossover agreement with ICC of the questionnaires by mode of administration
      Order of applicationICC (95% CI)
      EQ VASEQ-5D
      Paper-tablet0.75 (0.67–0.85)0.83 (0.75–0.89)
      Tablet-paper0.77 (0.68–0.84)0.76 (0.58–0.89)
      CI, confidence interval; EQ-5D, EuroQol five-dimensional questionnaire; EQ VAS, EuroQol visual analogue scale; ICC, intraclass correlation coefficient.

       Analysis of Mean Differences in the Crossover Group

      Analysis of variance was used to assess the mean and SD values of the VAS and EQ-5D in the crossover group, as shown in Table 3. The factors considered were the order of administration, type of application (tablet or paper), and the crossover group (paper-tablet or tablet-paper investigation). The P values were not significant for the order but were significant for the group in the VAS (P = 0.001) and the EQ-5D (P = 0.008). Regarding the type of application, the difference in means was significant for the VAS scores (P = 0.001). Analysis of the mean EQ-5D and VAS scores with respect to the type of application (tablet or paper) showed that the values were higher in the second application on paper and lower in the second tablet application. Regarding the group (paper-tablet or tablet-paper investigation), the mean EQ-5D and VAS scores were higher for the tablet-paper investigation.
      Table 3Mean VAS and EQ-5D scores in the crossover group for order, application type, and group
      ApplicationOrderEQ-5D, mean ± SDVAS, mean ± SDn
      Type of application
       PaperFirst0.85 ± 0.1477.5 ± 12.46118
      Second0.88 ± 0.1281.2 ± 12.76127
      Total0.87 ± 0.1379.4 ± 12.72245
       TabletFirst0.87 ± 0.1278.9 ± 12.73127
      Second0.85 ± 0.1476.6 ± 12.37118
      Total0.86 ± 0.1377.8 ± 12.58245
      Group
       Tablet-paper0.88 ± 0.1480.0 ± 12.77127
       Paper-tablet0.85 ± 0.1277.0 ± 12. 39118
      EQ-5D, EuroQol five-dimensional questionnaire; VAS, visual analogue scale.

       Analysis of Test-Retest Reliability for the Tablet Group

      According to the ICC, the correlation for the test-retest sample (tablet-tablet sample) was satisfactory for the VAS and very good for the utility score of the EQ-5D, as shown in Table 4.
      Table 4Test-retest agreement of the questionnaires by tablet mode
      Group of applicationICC (95% CI)
      EQ VASEQ-5D
      Tablet-tablet0.79 (0.66–0.87)0.85 (0.73–0.91)
      CI, confidence interval; EQ-5D, EuroQol five-dimensional questionnaire; EQ VAS, EuroQol visual analogue scale; ICC, intraclass correlation coefficient.
      Agreement and the κ coefficient were determined in the test-retest group for the five dimensions of the descriptive system of the EQ-5D, as shown in Table 5. Agreement among the five domains of the EQ-5D ranged from 86% to 95% and the κ coefficient was 0.69 for three dimensions, 0.79 for one dimension, and 0.81 for one dimension.
      Table 5Analysis of test-retest agreement summary scores for each dimension of the questionnaire
      EQ-5D dimensionAgreementκ coefficient (95% CI)
      Mobility95%0.81 (0.63–0.94)
      Personal care98%0.79 (0.33–1.00)
      Habitual activities93%0.69 (0.46–0.87)
      Pain/malaise84%0.69 (0.56–0.80)
      Anxiety/depression86%0.69 (0.64–0.96)
      CI, confidence interval; EQ-5D, EuroQol five-dimensional questionnaire.
      The reliability of the EQ-5D for the application of the questionnaire on paper or in tablet form was determined using the Cronbach α coefficient. Reliability was greater for the questionnaire on paper, with an α coefficient of 0.74 compared with 0.66 for the tablet.

       Analysis of the Differences in Test-Retest Mean Values

      For the test-retest of the tablet-tablet sample, analysis of the differences in mean values using a paired t test with 95% confidence interval showed that there were no significant differences in the utility scores of the EQ-5D and EQ VAS. These results demonstrate stability of the EQ-5D scores in the tablet method over repeated applications.

       Ease of Application and Acceptability of the Method

      In the comparison of the ease of application of the methods, most of the crossover group considered responding on the tablet to be easier than responding on paper. Of the group that first responded on the tablet, 43% considered responding on the tablet to be easier and 33% considered responding on paper and on the tablet to be equivalent. Of the group that first responded on paper, 46% considered the tablet method to be easier and 41% reported equivalent ease of application of the paper and tablet methods.
      The following results were obtained when the subjects were asked to comment about the ease of responding on paper or on the tablet: among the subjects who first responded on paper, 25% considered responding on the tablet to be very easy and responding on paper to be easy; 20% considered the two methods to be very easy and 47% considered them to be easy. Of the group that first responded on the tablet, 26% considered the method to be very easy and responding on paper to be easy; 19% considered responding to the two media to be very easy and 39% considered it to be easy. Thus, regardless of the order of application, 25% of the subjects considered responding on the tablet to be very easy and responding on paper to be easy.

      Discussion

      The present results provide an approach to measure quality of life using electronic media, which can be applied in clinical studies, economic analyses, and epidemiological studies. Investigators could apply the electronic instrument to different populations and compare the results, a fact that would represent an advance in the area of outcome measures for the assessment of health technologies and innovation to determine the effectiveness of various sanitary interventions.
      The present results also demonstrate that, in general, the application of the EQ-5D system using a digital device (tablet) offers similar psychometric properties to those of the application on paper, with equivalent results. Agreement analysis based on ICC demonstrated agreement in the test-retest group and in both crossover samples. In the test-retest, the VAS showed an ICC of 0.79, a higher value than that reported in the study by Bushnell et al. [
      • Bushnell D.M.
      • Reilly M.C.
      • Galani C.
      • et al.
      Validation of electronic data capture of the Irritable Bowel Syndrome–Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol.
      ], who obtained an ICC of 0.73 in the test-retest of the electronic device. In the present crossover group, the VAS showed an ICC of 0.75 and 0.77, in agreement with two studies that applied the crossover method to specific populations [
      • Bushnell D.M.
      • Reilly M.C.
      • Galani C.
      • et al.
      Validation of electronic data capture of the Irritable Bowel Syndrome–Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol.
      ,
      • Ramachandran S.
      • Lundy J.J.
      • Coons S.J.
      Testing the measurement equivalence of paper and touch-screen versions of the EQ-5D visual analog scale (EQ VAS).
      ].
      The present study tested the measurement equivalence of the EQ-5D on a tablet with the original Brazilian Portuguese paper version and detected no significant differences in the test-retest group. The crossover evaluation showed interaction between type of research and media, although the values did not appear to be clinically significant for the EQ-5D and VAS systems. This was in contrast to the study of Lundy and Coons [
      • Lundy J.J.
      • Coons S.J.
      Measurement equivalence of interactive voice response and paper versions of the EQ-5D in a cancer patient sample.
      ], with no significant effect of order or media. Current analyses demonstrated equivalence between methods when tested by both the ICC and the κ coefficient.
      In the present study, the test-retest κ values were higher for the mobility and personal care domains. Another study that assessed the EQ-5D equivalence in an interactive voice medium obtained a higher κ coefficient for the personal care and the anxiety/depression domains. The κ values obtained in the present study were slightly higher than those obtained for the interactive voice version, although both studies obtained κ values higher than 0.65 [
      • Lundy J.J.
      • Coons S.J.
      Measurement equivalence of interactive voice response and paper versions of the EQ-5D in a cancer patient sample.
      ,
      • Lundy J.J.
      • Coons S.J.
      Test-retest reliability of an interactive voice response version of the EQ-5D in a sample of cancer survivors.
      ]. In the present study, percent agreement was higher than 84% for all dimensions and the highest value (98%) was detected for personal care, whereas in the interactive voice study, agreement ranged from 85% to 98% for the dimensions of the EQ-5D [
      • Lundy J.J.
      • Coons S.J.
      Test-retest reliability of an interactive voice response version of the EQ-5D in a sample of cancer survivors.
      ].
      For the descriptive system, the present study obtained an excellent coefficient of 0.85 in the test-retest, a higher value than the ICC of 0.75 reported in similar studies using a voice command or electronic media [
      • Bushnell D.M.
      • Reilly M.C.
      • Galani C.
      • et al.
      Validation of electronic data capture of the Irritable Bowel Syndrome–Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol.
      ,
      • Lundy J.J.
      • Coons S.J.
      Test-retest reliability of an interactive voice response version of the EQ-5D in a sample of cancer survivors.
      ]. In the crossover application, the mean EQ-5D values were higher when the first application was on paper, with an ICC of 0.83 in the paper versus tablet application and of 0.76 in the tablet versus paper application, in agreement with a study of the EQ-5D with an electronic system, in which the ICC was 0.80 for the application on paper versus the electronic version and 0.77 for the application of the electronic version versus the application on paper [
      • Bushnell D.M.
      • Reilly M.C.
      • Galani C.
      • et al.
      Validation of electronic data capture of the Irritable Bowel Syndrome–Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol.
      ]. The study of Lundy and Coons [
      • Lundy J.J.
      • Coons S.J.
      Measurement equivalence of interactive voice response and paper versions of the EQ-5D in a cancer patient sample.
      ] conducted in 2011 detected higher correlations between applications, that is, an ICC of 0.89 for the paper versus voice command and an ICC of 0.88 for the voice command versus paper. Nevertheless, the cited study excluded from analysis subjects with a difference in EQ-5D scores higher than 2 SDs (0.28) between applications.
      The acceptability of the digital medium in the present study agrees with previous reports of equivalence of quality-of-life instruments using a digital system or paper. As also reported in other studies, the digital system was considered to be easier or as easy to use as the paper system by most of the subjects [
      • Bushnell D.M.
      • Martin M.L.
      • Scanlon M.
      • et al.
      Equivalence and measurement properties of an electronic version of the Psoriasis Symptom Inventory.
      ,
      • Chang Y.J.
      • Chang C.H.
      • Peng C.L.
      • et al.
      Measurement equivalence and feasibility of the EORTC QLQ-PR25: paper-and-pencil versus touch-screen administration.
      ]. In a Portuguese study that applied the short form 36 health survey to patients with autoimmune diseases, 84% of the participants preferred to respond to electronic media [
      • Ribeiro C.
      • Moreira L.
      • Silveira A.
      • et al.
      Development and use of touch-screen computer-assisted self interviewing in Portuguese patients with chronic immune diseases: evaluation of an electronic version of SF-36v2.
      ].
      The present study has some limitations such as the population being censored according to educational level, with the exclusion of illiterate subjects. For large-scale applicability of the electronic system to the Brazilian population, the sample should be expanded to other population groups.

      Conclusions

      The present analysis provided evidence that the scores obtained with the tablet version of the EQ-5D are equivalent to those obtained with the paper version. The tablet version was well accepted and showed agreement in the crossover and test-retest analyses.
      Source of financial support: This study was funded by National Council for Scientific and Technological Development (CNPQ), Brasília, Brazil. A. M. Bagattini received a graduate research scholarship from Coordination of Superior Level Staff Improvement (CAPES).

      References

        • Campolina A.G.
        • Ciconelli R.M.
        Qualidade de vida e medidas de utilidade: parâmetros clínicos para as tomadas de decisão [Spanish].
        Rev Panam Salud Publica. 2006; 19: 128-136
        • Lenert L.
        • Kaplan R.M.
        Validity and interpretation of preference-based measures of health-related quality of life.
        Med Care. 2000; 38: II138-II150
        • Drummond M.F.
        Methods for the Economic Evaluation of Health Care Programmes. 3rd ed. Oxford University Press, Oxford, UK2005
        • Szende A.
        • Oppe M.
        • Devlin N.J.
        EQ-5D Value Sets: Inventory, Comparative Review and User Guide.
        Springer, Dordrecht, The Netherlands2007
        • EuroQol Group
        EuroQol—a new facility for the measurement of health-related quality of life.
        Health Policy. 1990; 16: 199-208
        • Brooks R.
        • Rabin R.
        • de Charro F.
        The Measurement and Valuation of Health Status Using EQ-5D: A European Perspective.
        Kluwer Academic Publishers, Dordrecht, The Netherlands2003
        • Viegas Andrade M.
        • Noronha K.
        • Kind P.
        • et al.
        Societal preferences for EQ-5D health states from a Brazilian population survey. Value Health Reg.
        Issues. 2013; 2: 405-412
        • Santos M.
        • Cintra M.A.
        • Monteiro A.L.
        • et al.
        Brazilian valuation of EQ-5D-3L health states: results from a saturation study.
        Med Decis Making. 2016; 36: 253-263
        • Tosh J.C.
        • Longworth L.J.
        • George E.
        Utility values in National Institute for Health and Clinical Excellence (NICE) technology appraisals.
        Value Health. 2011; 14: 102-109
        • Weinstein M.C.
        • Siegel J.E.
        • Gold M.R.
        • et al.
        Recommendations of the panel on cost-effectiveness in health and medicine.
        JAMA. 1996; 276: 1253-1258
        • Brauer C.A.
        • Rosen A.B.
        • Greenberg D.
        • Neumann P.J.
        Trends in the measurement of health utilities in published cost-utility analyses.
        Value Health. 2006; 9: 213-218
        • Bushnell D.M.
        • Martin M.L.
        • Scanlon M.
        • et al.
        Equivalence and measurement properties of an electronic version of the Psoriasis Symptom Inventory.
        Qual Life Res. 2014; 23: 897-906
        • Gwaltney C.J.
        • Shields A.L.
        • Shiffman S.
        Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review.
        Value Health. 2008; 11: 322-333
        • Norman R.
        • King M.T.
        • Clarke D.
        • et al.
        Does mode of administration matter? Comparison of online and face-to-face administration of a time trade-off task.
        Qual Life Res. 2010; 19: 499-508
        • Coons S.J.
        • Gwaltney C.J.
        • Hays R.D.
        • et al.
        Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force Report.
        Value Health. 2009; 12: 419-429
        • Bushnell D.M.
        • Martin M.L.
        • Parasuraman B.
        Electronic versus paper questionnaires: a further comparison in persons with asthma.
        J Asthma. 2003; 40: 751-762
        • Bushnell D.M.
        • Reilly M.C.
        • Galani C.
        • et al.
        Validation of electronic data capture of the Irritable Bowel Syndrome–Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol.
        Value Health. 2006; 9: 98-105
        • Chang Y.J.
        • Chang C.H.
        • Peng C.L.
        • et al.
        Measurement equivalence and feasibility of the EORTC QLQ-PR25: paper-and-pencil versus touch-screen administration.
        Health Qual Life Outcomes. 2014; 12: 23
        • Ashley L.
        • Keding A.
        • Brown J.
        • et al.
        Score equivalence of electronic and paper versions of the Social Difficulties Inventory (SDI-21): a randomised crossover trial in cancer patients.
        Qual Life Res. 2013; 22: 1435-1440
        • Bushnell D.M.
        • Martin M.L.
        • Ricci J.F.
        • et al.
        Performance of the EQ-5D in patients with irritable bowel syndrome.
        Value Health. 2006; 9: 90-97
        • Ramachandran S.
        • Lundy J.J.
        • Coons S.J.
        Testing the measurement equivalence of paper and touch-screen versions of the EQ-5D visual analog scale (EQ VAS).
        Qual Life Res. 2008; 17: 1117-1120
      1. Pesquisa A-ABdEd. Critério de classificação econômica. Brazil, 2014.

        • Urbina S.
        Fundamentos da Testagem Psicológica.
        Artmed, Porto Alegre, Brazil2007
        • Switzer G.E.
        • Wisniewski S.R.
        • Belle S.H.
        • et al.
        Selecting, developing, and evaluating research instruments.
        Soc Psychiatry Psychiatr Epidemiol. 1999; 34: 399-409
        • Aaronson N.1
        • Alonso J.
        • Burnam A.
        • et al.
        Assessing health status and quality-of-life instruments: attributes and review criteria.
        Qual Life Res. 2002; 11: 193-205
        • Cronbach L.
        Coefficient alpha and the internal structure of tests.
        Psychometrika. 1951; 16: 297-334
        • Lundy J.J.
        • Coons S.J.
        Measurement equivalence of interactive voice response and paper versions of the EQ-5D in a cancer patient sample.
        Value Health. 2011; 14: 867-871
        • Lundy J.J.
        • Coons S.J.
        Test-retest reliability of an interactive voice response version of the EQ-5D in a sample of cancer survivors.
        Patient. 2012; 5: 21-26
        • Ribeiro C.
        • Moreira L.
        • Silveira A.
        • et al.
        Development and use of touch-screen computer-assisted self interviewing in Portuguese patients with chronic immune diseases: evaluation of an electronic version of SF-36v2.
        Acta Reumatol Port. 2010; 35: 208-214