Article Text

PDF

Patient assessment of treatment satisfaction: methods and practical issues
  1. D A Revicki
  1. Correspondence to:
    Dr D Revicki
    Center for Health Outcomes Research, MEDTAP International, 7101 Wisconsin Avenue, Suite 600, Bethesda, MD 20814, USA; Revickimedtap.com

Abstract

Measurement of treatment satisfaction in gastro-oesophageal reflux disease (GORD) is compromised by an insufficient conceptual foundation and poor assessment methods. The current state of the art in measuring treatment satisfaction is incomplete, and the existing measurement is insufficient. Here, the definition, conceptualisation, application, and methodological issues associated with measurement of treatment satisfaction in GORD are reviewed. Treatment satisfaction may be important for differentiating among GORD treatments, and for monitoring patient outcomes in clinical practice.

  • gastro-oesophageal reflux disease
  • patient reported outcomes
  • questionnaire
  • measurement instruments
  • treatment satisfaction
  • GORD, gastro-oesophageal reflux disease
  • PROs, patient reported outcomes
  • HRQoL, health related quality of life
  • SODA, severity of dyspepsia assessment
  • TSQ, treatment satisfaction questionnaire

Statistics from Altmetric.com

SUMMARY

Treatment satisfaction is a patient reported outcome that may give both useful insights into the patient’s perspective on their current treatment and differentiation among alternative treatments. Measurement of treatment satisfaction in gastro-oesophageal reflux disease (GORD) is compromised by an insufficient conceptual foundation and poor assessment methods. Treatment satisfaction is defined as the individual’s rating of important attributes of the process and outcomes of his/her treatment experience. Previous studies have demonstrated high rates of satisfaction with surgical treatments (85–92%) and medical treatments for GORD (70–94%) but most of these studies used simple global or single item measures. Use of attitude response scales and acquiescent response bias skew individual responses to satisfaction questions, inflate reliability estimates, and may bias treatment comparisons for older, low education, and low income subgroups. The current state of the art in measuring treatment satisfaction is incomplete, and the existing measurement is insufficient. Future instrument development for clinical trials requires more attention to conceptual issues, patient based qualitative research, and evaluation of psychometric qualities. Treatment satisfaction may be important for differentiating among GORD treatments, and for monitoring patient outcomes in clinical practice.

INTRODUCTION

Treatment satisfaction is one of the family of patient reported outcomes (PROs) that is potentially useful for understanding the patient’s perspective on their current treatment and that can differentiate among alternative treatments. Treatment satisfaction represents an important outcome as it is related to adherence and willingness to continue treatment. PROs include health related quality of life (HRQoL), self reported symptoms, functional status, and other end points derived from direct reports of patient experience. These PROs are frequently used to evaluate the impact of disease and treatments on the patient’s functioning, well being, and everyday life in clinical trials and other studies, including gastrointestinal disease.1 Measurement of treatment satisfaction in GORD, and for other gastrointestinal diseases, is compromised by an insufficient conceptual foundation and poorly developed assessment methods.

The primary objectives of this report are to review the definition, conceptualisation, application, and methodological issues associated with measurement of treatment satisfaction in GORD. This is not a comprehensive review of the literature on treatment satisfaction but examples are taken from the published literature to illustrate different issues and applications for clinical trials. Research on the relationship between treatment satisfaction and expectations, and demographic, physician, and health care system variables is summarised. Finally, comment is made on the use of satisfaction with treatment end points in making claims related to treatments for GORD.

DEFINITION AND CONCEPTUAL ISSUES

Treatment represents the application of different health care interventions for the cure or reduction of disease related symptoms. Treatment satisfaction is defined as the individual’s rating of important attributes of the process and outcomes of his/her treatment experience.2 Treatment satisfaction focuses on one aspect of satisfaction with medical care3–5 and involves the interaction of expectations, preferences, and satisfaction with medical treatment.6 Whereas health status instruments measure the outcomes of treatment (for example, biological/physiological, symptoms, functioning, and well being), treatment satisfaction scales assess the level of satisfaction with these health status outcomes.

Weaver and colleagues2 have provided a conceptual framework for understanding treatment satisfaction. This conceptual framework, in a somewhat modified form, is depicted in fig 1. Factors within the patient influence reports about satisfaction with treatment. Patient expectations, demographic characteristics, such as age and education, and personal preferences all affect treatment satisfaction. Expectations for treatment are clearly linked with perceptions of satisfaction.6 The pattern and characteristics (that is, side effects, effectiveness) of previous treatments for GORD, and the duration of GORD influence perceptions of treatment satisfaction. For example, patients treated with proton pump inhibitors may have high expectations for freedom from symptoms when prescribed new therapies. Clearly, the health care system, in general, influences satisfaction with medical care and treatment satisfaction.

Figure 1

Conceptual framework for treatment satisfaction (source: adapted from Weaver and colleagues2).

More important are the factors most directly connected to the treatment, and the way the treatment is delivered and directly experienced by the patient (see fig 1). Process factors cover issues associated with access to therapy, treatment related information, and cost (for example, prescription price or insurance co-pays). Outcomes of treatment represent an important determinant of treatment satisfaction, including impact on symptoms and adherence with the treatment regimen. For example, there is a strong relationship between treatment satisfaction and severity of GORD (see fig 2) where patients experiencing no or very mild symptoms are more satisfied compared with those with moderate or severe symptoms (Revicki, unpublished data, 2002). The complexity, discomfort, and convenience associated with treatment also affect patient perceptions and evaluations of satisfaction with the treatment. Finally, the individual’s intentions, as manifested by their willingness to continue therapy, or his/her preference for, or choice of, the treatment over other alternative treatments are relevant.

Figure 2

Treatment satisfaction by disease severity: gastro-oesophageal reflux disease (n = 689) (source: Revicki, unpublished data, 2002).

Several domains have been identified within the treatment satisfaction construct based on reviews of the literature2 and qualitative research (Coyne and Revicki, unpublished manuscript, 2002). A number of domains have been identified with treatment satisfaction, including treatment related effectiveness, discomfort, cost, regimen characteristics, and preference for continuing therapy (see table 1). In the review by Weaver and colleagues2 the domains most often included in treatment satisfaction measures cover overall satisfaction (47%), outcomes of treatment (47%), disease related information (37%), treatment related discomfort (31%), product design or appearance (31%), and convenience (31%).

Table 1

Treatment satisfaction domains

TREATMENT SATISFACTION INSTRUMENTS

Treatment satisfaction assessment in GORD, and gastrointestinal disease in general, is mostly limited to one or two simple questions on whether the patient is satisfied with the treatment, measured as “yes”, “no”, or “don’t know”, or by modified Likert scales ranging from “very satisfied” to “very dissatisfied”. The general psychometric literature7 and research on measuring satisfaction with medical care3,8,9 indicates that single item measures may be less reliable and sensitive to treatment differences than multi-item measures.

Treatment satisfaction studies in GORD

Several previous studies have demonstrated high rates of satisfaction with surgical treatments (85–93% satisfied) for GORD.10–15 Klingler and colleagues15 found that 92% of patients treated with laparoscopic antireflux surgery for oesophageal strictures refractory to medical therapy were satisfied with their treatment. In this study, satisfaction with therapy was measured using a single item on satisfaction that allowed responses of “yes”, “no”, or “indifferent”. These studies generally used simple global or single item measures, and no data on psychometric characteristics of the treatment satisfaction end points were provided.

Similar relatively high rates of satisfaction with medical treatments for GORD (70–94% satisfied) have also been observed in studies with cisapride,16 omeprazole,17 effervescent ranitidine tablets,18–21 lansoprazole,22,23 and rabeprazole.24 Bate and colleagues17 found that GORD patients treated with omeprazole reported significantly greater levels of treatment satisfaction compared with those treated with cimetidine. Effervescent ranitidine was preferred to standard ranitidine19 and famotidine wafers were preferred more than ranitidine tablets18 or effervescent ranitidine.20 Nelson and colleagues22 found that patients treated with omeprazole who were later converted to lansoprazole reported reductions in their satisfaction with their GORD treatment. Crawley and Schmitt,25 in a large survey of patients with self reported chronic heartburn, found that 45% of H2 receptor antagonist treated patients compared with 58% of proton pump inhibitor treated patients were totally satisfied with their therapy. Mathias and colleagues23 found that satisfaction with treatment was better for lansoprazole treated compared with ranitidine treated patients with GORD. The treatment satisfaction scale used in the study by Mathias and colleagues23 included four items covering how pleased they were with their medication, how much they benefited from their medication, whether they would recommend their treatment, and how they rated their current medication compared with previous medications. Global assessment of satisfaction with treatment was incorporated into a large open label study of rabeprazole in erosive oesophagitis.24 The results suggested that 88% of patients were satisfied with rabeprazole therapy but this study did not include a control group so it is difficult to interpret these findings.

As in the reflux surgery studies, most of the GORD medication studies used simple global or single item measures. No information is generally provided as to the measurement characteristics (that is, reliability, validity) of these measures. However, in the study by Mathias and colleagues,23 a multi-item treatment satisfaction scale was used with some evidence supporting reliability and validity.26

Instruments including treatment satisfaction

Several instruments are available that include treatment satisfaction items or subscales, including the gastro-oesophageal reflux data sheet,27 GORD-HRQL,26 and the severity of dyspepsia assessment (SODA).28 There are few data on the reliability and construct validity of these instruments (see table 2). More recently, the treatment satisfaction questionnaire (TSQ) was developed to measure treatment satisfaction in GORD (Coyne and Revicki, unpublished manuscript, 2002) based on a conceptual framework for treatment satisfaction and patient focus groups. The instrument contains seven subscales measuring symptoms, satisfaction, PRN, expectations, cost, and physician and bother domains of treatment satisfaction. The TSQ has good reliability (r = 0.66–0.96) and acceptable content validity, construct validity, and responsiveness.

Table 2

Available gastrointestinal disease related instruments with satisfaction measurement

In general, there is an absence of an underlying theoretical or conceptual model for the existing treatment satisfaction measures. Few of these instruments included qualitative data (for example, interviews, focus groups, cognitive interviewing) derived from patients to find out what domains are important to the consumers of GORD therapies as part of the development programme. The only multi-attribute treatment satisfaction instrument is the TSQ, and most of the existing measures consist of single item and global assessments. In addition, there are several measurement problems in many of these measures, including poorly worded items, acquiescent response bias, and response scales that may produce positive bias in patient reports on treatment satisfaction.

There are only limited data on psychometric qualities (that is, reliability, validity) of these measures, except for the GORD-HRQL, SODA, and TSQ. SODA assesses satisfaction with dyspepsia related health and not treatment satisfaction. TSQ has good reliability and is correlated with symptom severity, GORD symptoms, and disease specific and generic health status measures (Coyne and Revicki, unpublished manuscript, 2002). For example, the satisfaction and symptoms domains in the TSQ were more closed associated with days with GORD symptoms than the expectations domain of the TSQ (see fig 3). Other measures of treatment satisfaction have shown less consistent relationships with symptom severity, frequency, and reduction and effectiveness of treatment.

Figure 3

Mean treatment satisfaction questionnaire scores by days with symptoms (source: Coyne and Revicki, unpublished data, 2002).

MEASUREMENT METHODS

There are several key measurement problems associated with assessment of medical satisfaction and with treatment satisfaction.2,3,8,9,29,30 Most of the methodological research completed to date focuses on measuring satisfaction with medical care. Use of attitude response scales, acquiescent response bias, social desirability, and concern about negative consequences skew individual responses to satisfaction questions, inflate reliability estimates, and may bias treatment comparisons. Substantial acquiescent response bias is found across all methods for assessing patient satisfaction.8,9,31 For example, Ross and colleagues9 found that favourable responses to satisfaction evaluations ranged from 63% to 82% depending on the measurement method. Older respondents, those with less education, and those with poorer health were most highly acquiescent. Evaluative response scales (that is, “poor” to “excellent”) were less subject to acquiescent response bias than attitudinal scales (that is, “very satisfied” to “very dissatisfied”; “strongly agree” to “strongly disagree”), and multi-item measures that include balanced positively and negatively worded items were somewhat less subject to acquiescent response bias.8,9 Social desirability and concern about possible consequences of a negative evaluation affect patient ratings on satisfaction scales.30

Although most treatment satisfaction measures included in clinical studies are global single item scales, multi-item scales increase score variability and reliability, and are more sensitive to small to moderate clinical effects.3,8,9 More attention needs to be focused on documenting the psychometric characteristics of treatment satisfaction instruments, including the conceptual model, reliability, validity (that is, content, construct), responsiveness, and interpretation of meaningful changes in scores. There needs to be a better understanding of the cognitive processes underlying attitude measurements, such as treatment satisfaction, and qualitative research is needed to complete this research.32 Attention also needs to focus on variations by language translations and cultural adaptation and differences.

CONFOUNDING VARIABLES AND COVARIATES

Several demographic characteristics (that is, age, sex, education), health and symptom status, health care system variables (that is, physician-patient relationship, delivery of medical care, other health care attributes), patient expectations, and treatment history are associated with responses to medical and treatment satisfaction instruments. In terms of demographic characteristics, it has been found that patient satisfaction measures vary by age, education level, and income level.9,33 Treatment satisfaction ratings are more positive, in general, in the elderly, but evidence in less conclusive for sex and other demographic variables.

Several studies in other chronic diseases have found evidence for the relationship between effectiveness of therapy and patient ratings of treatment satisfaction.2 Marshall and colleagues34 found that patient satisfaction was significantly associated with mental health status but not physical health status. A more recent study (Coyne and Revicki, unpublished manuscript, 2002) found significant relationships between reflux symptoms, quality of life in reflux and dyspepsia (QOLRAD) scale total scores, SF-36 physical and mental health scores, and TSQ symptom and satisfaction scores (see table 3). In addition, there are a number of clinical studies demonstrating the implied or direct relationship between absence of GORD related symptoms and treatment satisfaction.16–24 Kurata and colleagues35 found that satisfaction with medical care among dyspepsia patients was associated with improvements in symptom response, while prescribed medication was not associated with satisfaction.

Table 3

Correlations between treatment satisfaction questionnaire, symptoms, and health related quality of life

Aspects of the patient-physician interaction and relations impact on measures of satisfaction with medical care36 and are likely to be associated with treatment satisfaction. The quality of the interaction, physician expressed effect, communication style, and level of information provision all impact ratings of satisfaction. Patient expectations are closely associated with their ratings of satisfaction with medical care.6,37,38 In general, when patient’s expectations are met within the medical encounter, they report more satisfaction with medical care. Health system characteristics impact on patient satisfaction with medical care and these characteristics are likely to affect patient satisfaction with treatment.

REGULATORY ISSUES FOR TREATMENT SATISFACTION CLAIMS

Measures of treatment satisfaction are important secondary end points for evaluating the outcomes of new treatments for GORD. Given the association between treatment satisfaction and adherence and willingness to continue taking a treatment, this outcome measure may provide insight into patient behaviour related to achieving the full effects of the treatment. To our knowledge, there are no examples of the use of treatment satisfaction measures in evaluating medical treatments and in obtaining regulatory claims for GORD treatments. Given the interest in PROs and the focus of the pharmaceutical industry in demonstrating product value, in terms of patient outcomes, treatment satisfaction end points may receive more attention.

It is likely that the USA Food and Drug Administration will review treatment satisfaction claims based on the same criteria used for other PRO claims, such as HRQoL.39,40 The European Authorities do not have a policy for treatment satisfaction outcomes and will probably evaluate these claims based on data submitted by industry sponsors.41 Any approved claim will require demonstrated safety and clinical efficacy before any consideration of PROs, such as treatment satisfaction. Treatment satisfaction claims will need to be based on scientifically adequate and well controlled study designs, psychometrically sound instruments, appropriate and a priori declared statistical analysis methods, and complete and fair reporting of findings. It will also be important to clearly delineate guidelines for interpreting minimally important differences or effects for treatment satisfaction instruments.

SUMMARY

In summary, the current state of the art in measuring treatment satisfaction is incomplete and the existing measurement is insufficient. Research suggests that patients report high rates of satisfaction with surgical treatments (85–92%) and medical treatments for GORD (70–94%). The existing published studies use single item or global measures, and multi-item multidomain treatment satisfaction measures are needed to assess outcomes of GORD treatments. Methodological problems in assessing treatment satisfaction, related to acquiescent response bias and use of attitude response scales, skew individual responses to satisfaction questions, inflate reliability estimates, and may bias treatment comparisons for older, low education, and low income subgroups. Willingness to pay studies may be helpful in understanding important treatment characteristics for GORD from the patient’s perspective,41 although Ross and colleagues9 found that these methods may not be useful for measuring satisfaction with medical care.

Future instrument development for clinical trials requires more attention to conceptual issues, patient based qualitative research, and evaluation of psychometric qualities. More data are needed on how to interpret scores on treatment satisfaction measures, and on identifying minimally important differences or changes in these scores. Treatment satisfaction may be important for differentiating among GORD treatments and for monitoring patient outcomes in clinical practice, but the current status of measurement needs improvement.

REFERENCES

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles